Evaluation of Givewell Blogs | GiveWell

You are here

Evaluation of Givewell Blogs

Subscribe to Evaluation of Givewell Blogs feed
Exploring how to get real change for your dollar.
Updated: 30 min 34 sec ago

Update on GiveWell’s web traffic / money moved: Q1 2016

Tue, 07/19/2016 - 15:21

In addition to evaluations of other charities, GiveWell publishes substantial evaluation of ourselves, from progress against our goals to our impact on donations. We generally publish quarterly updates regarding two key metrics: (a) donations to top charities and (b) web traffic (though going forward, we may provide less frequent updates).

The tables and chart below present basic information about our growth in money moved and web traffic in the first quarter of 2016 compared to the previous two years (note 1).

Money moved and donors: first quarter

Money moved by donors who have never given more than $5,000 in a year increased about 50% to $1.1 million. The total number of donors in the first quarter increased about 30% to about 4,500 (note 2).

Most of our money moved is donated near the end of the year (we tracked 70% or more of our total money moved in the fourth quarter each of the last three years) and is driven by a relatively small number of large donors. Because of this, we do not think we can reliably predict our growth and think that our year-to-date total money moved provides relatively limited information about what our year-end money moved is likely to be (note 3). We therefore look at the data above as an indication of growth in our audience.

Web traffic through April 2016

Growth in web traffic excluding Google AdWords increased 10% in the first quarter. GiveWell’s website receives elevated web traffic during “giving season” around December of each year. To adjust for this and emphasize the trend, the chart below shows the rolling sum of unique visitors over the previous twelve months, starting in December 2009 (the first period for which we have 12 months of reliable data due to an issue tracking visits in 2008).

We use web analytics data from two sources: Clicky and Google Analytics (except for those months for which we only have reliable data from one source). The raw data we used to generate the chart and table above (as well as notes on the issues we’ve had and adjustments we’ve made) is in this spreadsheet. (Note on how we count unique visitors.)


Note 1: Since our 2012 annual metrics report we have shifted to a reporting year that starts on February 1, rather than January 1, in order to better capture year-on-year growth in the peak giving months of December and January. Therefore, metrics for the “first quarter” reported here are for February through April.

Note 2: Our measure of the total number of donors may overestimate the true number. We identify individual donors based on the reported name and email. Donors may donate directly to our recommended charities and not opt to share their contact information with us, or donors may use different information for subsequent donations (for example, a different email), in which case, we may mistakenly count a donation from a past donor as if it was made by a new donor. We are unsure but would guess that the impact of this issue is relatively small and that the data shown are generally reflective of our growth from year to year.

Note 3: In total, GiveWell donors directed $2.6 million to our top charities in the first quarter of 2016, compared to $2.0 million that we had tracked in the first quarter of 2015. For the reason described above, we don’t find this number to be particularly meaningful at this time of year.

Note 4: We count unique visitors over a period as the sum of monthly unique visitors. In other words, if the same person visits the site multiple times in a calendar month, they are counted once. If they visit in multiple months, they are counted once per month.

The post Update on GiveWell’s web traffic / money moved: Q1 2016 appeared first on The GiveWell Blog.

GiveWell’s money moved and web traffic in 2015

Fri, 05/13/2016 - 13:56

GiveWell is dedicated to finding outstanding giving opportunities and publishing the full details of our analysis. In addition to evaluations of other charities, GiveWell publishes substantial evaluation of our own work. This post lays out highlights from our 2015 metrics report, which reviews what we know about how our research impacted donors. Please note:

  • We report on “metrics years” that run from February through January; for example, our 2015 data cover February 1, 2015 through January 31, 2016.
  • We differentiate between our traditional charity recommendations, our work on the Open Philanthropy Project, and other charitable giving.
  • More context on the relationship between Good Ventures and GiveWell can be found here.

Summary of influence: In 2015, GiveWell influenced charitable giving in several ways. The following table summarizes our understanding of this influence.

Total money moved: In 2015, GiveWell tracked $110.1 million in money moved to our recommended charities. Our money moved only includes donations that we are confident were influenced by our recommendations.

Open Philanthropy Project: As part of our work on the Open Philanthropy Project, we advised Good Ventures to make grants totaling $6.4 million. This was in addition to Good Ventures’ support for our recommended charities.

Money moved by charity: Our four top charities received the majority of our money moved. Our four standout charities received a total of $2.2 million.


Money moved by size of donor: In 2015, the number of donors and amount donated increased across each donor size category. In 2015, 95% of our money moved (excluding Good Ventures) came from about 15% of our donors, each of whom gave $1,000 or more.


Donor retention: The total number of donors who gave to our recommended charities or to GiveWell unrestricted increased about 60% year-over-year to 15,274 in 2015. This included 10,669 donors who gave to our recommended charities for the first time. Among all donors who gave in the previous year, about 40% gave again in 2015, up from about 33% who gave again in 2014.


Our retention was stronger among donors who gave larger amounts or who first gave to our recommendations prior to 2013. Of larger donors (those who gave $10,000 or more in either of the last two years), about 80% who gave in 2014 gave again in 2015.


GiveWell’s expenses: GiveWell’s total operating expenses in 2015 were $3.4 million. Our expenses increased from about $1.8 million in 2014 as the size of our staff grew, largely to support the Open Philanthropy Project. We estimate that about one-third of our total expenses ($1.1 million) supported our traditional top charity work and about two-thirds supported the Open Philanthropy Project. In 2014, we estimated that expenses for our traditional charity work were about $900,000.

Donations supporting GiveWell’s operations: Prior to 2013, GiveWell relied on a small number of donors to provide unrestricted support for our operations. Since 2013, we have asked more donors to support our operational costs and asked donors to support us at a higher level than we had in previous years. In 2015, we raised $4.9 million, up from $3.0 million in 2014. Several institutions and the six largest individual donors contributed about two-thirds of GiveWell’s funding in 2015.


Donor demographics: We continued to collect information on our donors. We found the picture of our 2015 donors to be broadly consistent with previous information. Based on reports from donors who gave $2,000 or more, we found:

  • The most common ways that donors found us were via Peter Singer and personal referrals.
  • About 70% of our donors are under 40, and about 60% work in technology or finance.

Web traffic: Unique visitors to our website increased by 12% in 2015 compared to 2014 (when excluding visitors driven by AdWords, Google’s online advertising product).

For more detail, see our full metrics report (PDF).

The post GiveWell’s money moved and web traffic in 2015 appeared first on The GiveWell Blog.

GiveWell research plans for 2016

Wed, 04/06/2016 - 13:53

Over the past couple of years, we’ve put a lot of effort into hiring and training staff and we now have significantly more capacity to do research than we have in past years. Some of our increased capacity will support the Open Philanthropy Project, which we hope will be a separate organization by the end of 2016; its plans for the year will be discussed on the new Open Philanthropy Project blog. We also expect to have more capacity for GiveWell’s work of finding outstanding evidence-based charities.

At the same time, we have come to believe that the kind of work we’ve recently been doing to find top charities – deeply investigating the most promising-seeming charities we know of, based largely on which interventions they carry out – has limited promise. In past years – and at the beginning of this year – we hoped that these investigations would lead relatively quickly to new top charities. Now, we believe that we’ve already (previously) identified most of the strongest charities by our criteria, and there aren’t many strong candidates left (though there are a few that we continue to investigate, and we remain willing and eager to investigate further promising groups if we come across them). With that in mind, we have begun seeing more potential in other research priorities, such as supporting the development of new organizations and new evidence bases.

A future post will elaborate on why we’ve formed this view. This post focuses on laying out our plans for GiveWell’s research work in 2016, taking this view into account.

In brief, in 2016:

  • We plan to focus much of our capacity on a small number of initiatives that are unlikely to result in new top charities in 2016, but which we hope will lead to new top charities that are competitive with our current top charities in 2017 or 2018.
  • We plan to intensify our work following our current top charities and are tentatively planning to make site visits to distributions funded by the Against Malaria Foundation and work supported by Evidence Action’s Deworm the World Initiative.
  • We are also planning a substantial project focused on the question of whether or not we should recommend that Good Ventures give significantly more than it has in the past to support insecticide-treated nets, arguably the most promising area we know of for substantial additional funding.
  • We also hope to take on additional work (described in detail below) but plan to prioritize this work below the items listed above.
  • We plan to put more staff time into donor outreach than we have in the past and discuss our priorities for that work below.

This plan represents a significant shift from previous years, when our primary goal was improving the list of top charities we published at the end of each year. We plan to write more about the reasoning behind this shift in a future post.

What we’ve done so far this year

In January and February 2016 we:

  • Put significantly more effort into getting input on our plans from non-senior staff than we have in past years. To start, we asked staff and a small number of GiveWell followers to make probabilistic predictions about which charities would become top charities if we investigated them this year. The results of this exercise led to this initial list of possible priorities (listed in no particular order). This list represents the organizations that we would guess are most likely to become top charities at the end of 2016. When we later revised our plan, we held a series of staff meetings to discuss details of the plan and what the new plan might be missing.
  • Had exploratory conversations with several charities on that list and others in the field of global health and development. The goal of conversations with charities was to explain our application process and ask them basic questions about their programs, monitoring and evaluation, and need for additional funding. The goal of conversations with others working in global health and development was to generate a list of additional organizations to contact.

Ultimately, this work made us more pessimistic that prioritizing work on all the organizations listed above would lead to new top charities by the end of 2016, as discussed in the introduction of this post, and we refined our plans for the year as a result. A future post will elaborate on this development.

Top priorities for research

Our top priorities are:

  • Supporting the development of potential future GiveWell top charities: making grants to organizations that could become top charity contenders in the future or supporting research that could lead to more organizations that are a strong fit with our criteria. This work is unlikely to result in new top charities in 2016, but we hope it will lead to new top charities that are competitive with our current top charities in 2017 or 2018. This work might include:
    • Providing early stage funding to organizations that aim to scale up programs with strong evidence of effectiveness and cost-effectiveness. (For example, New Incentives or Evidence Action’s No Lean Season program.)
    • Funding research on programs that are candidates to become priority programs. (For example, this grant to support research on an incentives for immunization program.)
    • Funding organizations that run priority programs to increase or improve their monitoring, or funding a third party to do this monitoring.
  • Considering additional funding for insecticide-treated nets: A significant funding gap exists for insecticide-treated nets, and this gap appears to be as cost-effective an opportunity as any other we have found. This project involves determining whether there are high quality opportunities to provide significantly more funding for insecticide-treated nets than we have in the past. It will involve conversations with the major bednet funders (e.g., Global Fund to fight AIDS, Tuberculosis, and Malaria and the President’s Malaria Initiative) and others familiar with how to identify funding gaps for bednets and what the options are for monitoring distributions. We have also been discussing with the Against Malaria Foundation (AMF) what it would take to quickly scale up AMF’s work. The goal of this work is to identify additional funding opportunities for funding insecticide-treated nets in 2016.
  • Intervention prioritization: quick investigations on a large number of interventions with the goal of finding more priority programs. We have looked at many interventions historically, but regularly learn of programs that we do not know very much about.
  • Current top charities: continuing to follow our current top charities and trying to answer our highest priority unanswered questions about these groups. More on this below.
  • New evidence on deworming and bednets. The next round of follow up on a key deworming study is expected to be available later this year and could make a big difference to our view of deworming. We’re also looking more into the degree to which insecticide resistance may be reducing the impact of bednets.

Other research we will undertake if we have the time to do so

  • Micronutrient fortification charities. Last year, we tried but were unable to find compelling evidence that the Iodine Global Network (IGN) or the Global Alliance for Improved Nutrition (GAIN) had successfully contributed to the impact of salt iodization programs (write-ups forthcoming). We also began investigating Project Healthy Children. We may continue some of these investigations this year and have also reached out to the Micronutrient Initiative and the Food Fortification Initiative.
  • Neglected tropical disease (NTD) charities. We began investigating Sightsavers and END Fund’s work on deworming last year and may continue with those organizations this year and expand the investigations to cover multiple NTDs. We have reached out to the Carter Center and Helen Keller International (HKI) about their NTD programs. HKI declined to participate at this time.
  • Surgery charities. We have had several conversations with organizations that work on cataract surgery and we may reach out to organizations that work on obstetric fistula surgery. Our initial impression from these conversations is that it will be very challenging to understand the impact that these charities’ programs have. We may also consider other surgical interventions (such as trachoma).
  • Other organizations. If organizations apply for a recommendation and seem sufficiently promising, we will aim to review them.
  • Publishing research we largely completed in 2015: updates on standout charities (GAIN, IGN, and Living Goods), interim reviews of charities we began investigating in 2015 (Sightsavers, END Fund, and Project Healthy Children), and intervention reports (folic acid fortification, surgery for cataracts, trachoma and fistula, measles immunization campaigns, mass drug administration for lymphatic filariasis, and “Targeting the Ultra Poor”).

Research we considered but do not expect to undertake

The following investigations are ones that we considered doing this year but don’t currently expect to get to. This could change if some of the higher priority work turns out to be less promising than expected.

  • Mega-charities. We could try to work with one or more large organizations with very diverse programs to figure out how to scale-up work on one of our priority programs.
  • Charities that work on programs that are probably more cost-effective than cash transfers but not by a large enough margin that it seems worth highly prioritizing work on them.
    • Voluntary medical male circumcision. We are interested in talking to PSI, the only major organization we know of working on this program, but do not plan to prioritize this program beyond that.
    • “Targeting the ultra poor” or “graduation” programs.
    • Lymphatic filariasis.
    • Incentives for immunization. We previously funded research on this program and have been working with IDinsight on a cost-effectiveness analysis.
  • Immunization programs. We have put in a fair amount of work into looking for room for more funding for scaling up immunization programs and have largely failed to find opportunities (2012 write-up; recent example).

More detail on potential further research on current recommended charities

One of our top priorities for 2016 is continuing to follow our current top charities and trying to answer our highest priority unanswered questions about these groups. We moved over $100 million to these groups in 2015 and whether we recommend a similar (or greater) level of support in 2016, and how we recommend allocating funds among them, depends on answering: (1) what is our best estimate of the organization’s impact and cost-effectiveness? and (2) how much room for more funding do they have?

Top charities

In past years, we’ve updated our top charity reviews once a year, in November. This year, we plan to refresh these reviews twice, in June and November. As we have at the end of the year, we expect to reconsider what recommendation we make to donors about how to allocate donations amongst our top charities in June.

Summary of our research plans for each of our top charities (note that the strategy documents were written in February):

  • Against Malaria Foundation (AMF). We plan to follow AMF’s progress closely in 2016. Key questions include (a) how quickly is AMF committing funding to new distributions, and (b) can we get a more detailed understanding of how data is collected in pre- and post-distribution surveys. More details here.
  • Schistosomiasis Control Initiative (SCI). The amount of time we spend on SCI this year depends on whether we see a significant improvement in the quality of SCI’s financial information (how it has spent funds, how much funding it holds, and projected expenses). If it does not improve, we will likely deprioritize much additional work on SCI. If it does, we would be interested in exploring the research questions detailed here.
  • Deworm the World Initiative. Of our top charities, we feel that there is the largest gap between what we could know and what we do know for Deworm the World. In particular, we’ve focused on Deworm the World’s work in India, because in the past most unrestricted funds were used in India. Going forward, unrestricted funds will largely be used in new programs. We aim to follow Deworm the World’s progress in new countries closely and to better understand its past work by learning more about its program in Kenya. Details here.
  • GiveDirectly. Our main goals from following GiveDirectly are to see if the quality of monitoring remains high, it is able to enroll new recipients quickly, and we can learn more about the impact of its work with partners to make cash a baseline against which other development programs are judged. Details here.

Standout charities

  • Development Media International. We’re not planning to consider DMI as a possible top charity in 2016. The results from a randomized controlled trial (RCT) of its program that DMI shared last year were not in line with what we would have wanted to see for DMI to become a top charity. More recently, DMI shared some additional results from the RCT (which are not yet public). We believe that taken together these results provide conflicting evidence for DMI’s impact. DMI stands out for its commitment to transparency and rigorous evaluation and we will consider working with DMI to continue to build the evidence base around behavior change through mass media. We see this as a long-term project that is unlikely to result in DMI’s being a top charity in 2016.
  • Iodine Global Network. We are planning to follow up with IGN about a few case studies that IGN thought might provide additional evidence of its impact.
  • The Global Alliance for Improved Nutrition (GAIN) – Universal Salt Iodization (USI) program. We’re not planning to consider GAIN’s USI program as a possible top charity in 2016. We have not been able to establish clear evidence of GAIN successfully contributing to the impact of iodization programs, and think it is unlikely that more work on this will be useful.
  • Living Goods. It’s fairly unlikely that we will consider Living Goods as a possible top charity in 2016. We would revisit this if we were to see significant improvements in the rigor of Living Goods’ monitoring or if we significantly changed our cost-effectiveness estimate for its work.

Plans for donor outreach

We have not historically prioritized outreach at GiveWell, instead choosing to devote staff capacity primarily to our research work. Now, with the addition of new research staff as well as the continued growth of GiveWell’s donor base, we feel it is appropriate to dedicate more capacity to outreach for GiveWell in service of our mission to make our research available to help individuals decide where to give.

In 2016, we plan to have 1.5 staff members devoted to outreach related to GiveWell and the Open Philanthropy Project. Due to this being early on in our outreach work, we’re tentatively planning to reassess our priorities every month for the first half of the year, and then every quarter. As of the publication of this blog post, we expect the following to be top priorities for GiveWell outreach in 2016:

  • Donor calls and meetings. We expect that connecting with individuals who have donated to GiveWell will be an important part of our outreach going forward, although as we’re relatively new to prioritizing this, we plan to survey donors about whether this is something that they find useful. We’re hoping to learn more about the donors who use our work and any questions or feedback they have, as well as to offer an opportunity for donors to stay up to date on GiveWell’s work. More here.
  • Launching a redesigned website. The redesign will largely improve the look and feel of the site with some minor improvements in navigation and content organization.
  • Improving GiveWell’s written communications. This includes:
    • Revisiting and refreshing content on our website (e.g., a recent update to our criteria page) to ensure it’s up-to-date and clearly presented, particularly for individuals who aren’t familiar with our research.
    • Publishing content to our blog, in the hopes of highlighting research and providing additional insight into our values, process, and findings. We will need to put more effort into writing blog posts in order to maintain our previous pace of about one blog post per week, since many types of blog posts that previously appeared here will now be appearing on the Open Philanthropy Blog.

The post GiveWell research plans for 2016 appeared first on The GiveWell Blog.

GiveWell’s progress in 2015

Tue, 04/05/2016 - 11:52

This post reviews and evaluates last year’s progress on our traditional work of finding and recommending evidence-based, thoroughly vetted charities that serve the global poor. It has two parts. First, we look back at the plans we laid out in early 2015 and compare our progress against them, providing details on some of the most significant accomplishments and shortcomings of the year. Then, we reflect on the overall impact of our traditional work and critically evaluate some of our major strategic decisions. In our next post, we will cover our plans for GiveWell’s work in 2016.

SummaryIn brief, when evaluating ourselves against the goals we laid out in early 2015, we feel that we broadly achieved our primary goals for the year while we generally fell short on several of our secondary goals.

The overall impact of GiveWell’s recommendations continued to increase substantially in 2015, as we tracked more than $100 million that was donated to our recommended charities as a direct result of our research.

This self-evaluation post focuses primarily on how we have grown as an organization and the recent strategic decisions we have made since these are most relevant to thinking about the impact of recent GiveWell-focused work.

Our progress in 2015 relative to our plansIn our “2015 plan” blog post, we wrote:

This year, our primary goals are to:

  • Build management and research capacity for GiveWell’s traditional work while further reducing senior staff time (note 1) spent on this work, primarily by reallocating Elie Hassenfeld’s management responsibilities related to GiveWell’s traditional work.
  • Maintain our core research product by completing updates on all eight 2014 recommended charities and determining which of them should be recommended as top charities for the 2015 giving season.

Our secondary goals for 2015 are to:

  • Continue to seek outstanding giving opportunities by reviewing 2-4 new charities and publishing 2-4 new intervention reports.
  • Improve the cost-effectiveness analyses and room for more funding analyses in charity reviews.
  • Finish and launch a redesigned GiveWell website.
  • Make further progress on experimental work to “seed” potential recommended charities.We expect our total output on “top charities” work to be roughly comparable to last year’s, despite a growing staff, because (a) a major focus of the coming year is training, and we expect to trade some short-term efficiency for long-run output; (b) we may be reallocating some capacity from our “top charities” work to the Open Philanthropy Project this year.

We feel that we broadly achieved our primary goals for the year, while we fell short on several of our secondary goals.

Goals that we feel we accomplished include:

  • Building management and research capacity for GiveWell’s traditional work while further reducing the time that Elie spent on this work. Specifically:
    • All GiveWell staff track how they spend time at work. In 2014, Elie spent 36% of his time on GiveWell’s traditional work, and in 2015, Elie spent 26% of his time on this work.
    • In 2014, Elie was primarily responsible for managing all research staff. In 2015, Natalie Crispin and Josh Rosenberg took on research management responsibilities, and at the end of the year, managed 10 research staff between them. Timothy Telleen-Lawton and Eliza Scheffler also took on management responsibilities, primarily for operations-focused staff.
    • Most of the other staff working on GiveWell’s traditional work in 2015 were relatively new. Because they were new, most of the time they spent was focused on training. We discuss staff time allocations in greater detail below.
  • Maintaining our core research product. We completed and published updates on all four top charities from 2014. We also conducted deeper investigations of 3 out of 4 of our standout charities (Global Alliance for Improved Nutrition (GAIN), Iodine Global Network (IGN), and Development Media International (DMI)), as planned, in an effort to determine whether they should be recommended as top charities for the 2015 giving season. We ultimately did not recommend any of these charities as top charities. We did not publish our updates on GAIN and IGN, and those are still forthcoming.
  • Improving our room for more funding analyses in charity reviews, which we wrote about extensively in our November 2015 blog post announcing our updated recommendations.
  • Continuing to make limited but steady progress on experimental work to “seed” potential recommended charities via grants to Evidence Action’s No Lean Season Program, New Incentives, and two randomized controlled trials focused on incentives for immunization.

Areas in which we fell short include:

  • We did not publish updates (or even reach tentative conclusions internally) about any new potential recommended charities, and we published only 2 new intervention reports. We engaged with Project Healthy Children, UNICEF, The END Fund, Episcopal Relief & Development’s NetsforLife® Program, and Sightsavers. (We published a page on Children Without Worms, but it was extremely short.) We published 2 new intervention reports — on vitamin A supplementation and tetanus immunization campaigns — but both of these largely relied on work we had completed in 2014.
  • As discussed in our November 2015 blog post announcing our updated recommendations, we felt more confident in our cost-effectiveness analyses (CEAs) at the end of 2015 than we had in previous years, and we made some of the improvements we planned. Nevertheless, we don’t believe that we made as much progress as we had hoped for when we set the goal to improve our CEAs in early 2015. In particular, key inputs and judgment calls in our CEA were decided on by different staff members at different points in time, which made it more difficult than ideal for individual staff members to understand all details of the CEA. We hope to resolve this issue by making at least one staff member responsible for fully understanding and explaining all aspects of our core CEAs in the future.
  • We did not finish and launch a redesigned GiveWell website. The GiveWell website is large and it took longer than the firm we worked with expected to complete a version that was working properly. The firm we contracted with also built the Open Philanthropy Project website, and in September 2015, we explicitly prioritized completing the Open Philanthropy Project (“Open Phil”) site before the GiveWell site.

Overall, we succeeded in passing significant responsibilities from Elie to others, but we saw somewhat less overall research output than we had hoped for. Part of this was because, in the process of transferring responsibilities, we made some mistaken decisions – i.e., in some cases staff put substantial work into assignments that we ultimately determined were not a fit for them.

Other self-evaluation questionsWhat was GiveWell’s overall impact in 2015?

The overall impact of GiveWell’s recommendations continued to increase substantially in 2015. We tracked more than $100 million that was donated to our recommended charities as a direct result of our research. Excluding Good Ventures’ giving, we moved more than $30 million to our recommended charities. (More details on our 2015 money moved will be in our forthcoming 2015 metrics blog post.) This was a major increase relative to 2014, when we tracked about $27.8 million in money moved to our recommended charities, with about $13 million coming from non-Good Ventures donors.

Does our impact justify our staff size, in absolute terms?

As mentioned above, in 2015 we moved more than $100 million to our recommended charities. Over the same period, we spent approximately $3.8 million on our operations, of which $1.3 million was spent on GiveWell’s traditional work and $2.5 million on the Open Philanthropy Project.

We previously wrote that we believe that expenses that are 15% of money moved are well within the range of normal, so we feel comfortable with the relative size of our operating expenses at this point.

Is recent growth in staff justified? Has growth in output matched growth in staff?

Staff size has grown significantly over the past few years: from 11 people at the end of 2013 to 32 at the end of 2015. We think it is reasonable to question whether this growth has been justified and productive, in light of the fact that:

  • We haven’t significantly increased the number of new major reports (charities or interventions reviewed in depth). We estimate that we published 4 major new reports in 2013, 5 in 2014, and 6 in 2015 (details on our rough estimate in this Google sheet including year-to-date information for 2016).
  • Most of our 2015 impact (in terms of money moved) likely could have been achieved with reduced research work in 2015. For example, if we had only stayed up-to-date on our past top charities, we would have been in a position to have the same top charities list as what we ultimately published at the end of 2015. The bulk of additional research work that we did in 2015 did not seem to have significant direct impact, largely because the new giving opportunities that we investigated did not lead to new top charities, though it would have been difficult to confidently predict that this would happen in advance (we reflect on whether we chose the best research priorities in 2015 below).

On the other hand:

  • Since 2013, we have greatly expanded the Open Philanthropy Project, which is currently formally housed at GiveWell though we hope to separate the organizations in 2016. Much of our increase in staff size has been for the Open Philanthropy Project.
  • Many of our most senior staff have switched over to working primarily on the Open Philanthropy Project, and senior staff are particularly challenging to replace. We don’t think we could have maintained the same output if we had simply hired one new person for each senior staff member who switched over.
  • Capacity building – hiring, training and evaluating new staff – is itself a time-consuming and long-term project. In addition, increased staff size creates the need for more work in and of itself. For example: (1) in 2015, we outgrew our office and had to find new office space. Finding new office space was only necessary because of our staff size. (2) Because our staff is larger and we have a larger office, we now need an office manager. (Early in our history, we didn’t have an office, or we worked in a shared workspace where someone else played this role.)
  • As our money moved grows, it becomes more worthwhile to be more thorough (as we believe we have been), and it becomes more worthwhile to try to find new top charities even if doing so is a long-term and uncertain proposition. While the work we did in 2015 did not lead to any immediate new top charities, we believe that some of it may have significant returns in the future (e.g., the “GiveWell experimental” work discussed above and the preliminary work that we did on new charities and interventions that we have not finished evaluating).

We previously discussed our reasoning about why we believe it is worthwhile to continue to expand GiveWell’s research capacity despite limited recent returns in last year’s self-evaluation post. This year, we decided to take a more in-depth look at specifically where our increased capacity has gone, while adjusting for staff seniority. Specifically, we roughly estimated the number of full-time, co-founder-equivalent people working on each area in each year. (Co-founder time was given a score of 1 but time spent by newer staff was scaled down significantly; for example, we multiplied time spent by entry-level staff in their first year at GiveWell by 0.1. These figures are far from precise, and are intended only to give a rough guess at our changes in total capacity over time.)

We’ve divided our work into several categories:

  • GiveWell’s traditional research: time spent on evaluating new charities or interventions and updating our work on existing top charities or interventions.
  • Open Phil: time spent on work related to the Open Philanthropy Project.
  • Other output: work that directly furthers GiveWell’s and the Open Philanthropy Project’s missions but isn’t research related. Among other things, this includes work related to donation processing (e.g., opening the mail, entering donations into our database, sending thank you receipts), donor communication (answering donor questions, staying in touch with larger donors), and general outreach (giving talks, speaking to the media, and improving our web content). All of these categories of work have increased significantly as our public profile has grown.
  • Overhead: work necessary due to the increased size of our staff, such as changing offices and having an office manager (as mentioned above).
  • Capacity building: time spent by relatively senior staff to increase our future capacity. This includes time spent (a) recruiting (e.g., reviewing resumes, interviewing and evaluating candidates) and (b) training and evaluating new staff with the goal of increasing future capacity, not generating additional short-term research output. For example, in 2015, Elie estimates that he spent approximately 60% of his time building capacity.
  • Training: time spent by newer staff being trained. These staff members are primarily working on projects where the goal is training and evaluation, not short-term research output.

The table shows that our overall capacity has increased significantly since 2012 (from approximately 3 co-founder equivalents then to 9 today). Most of that increase has gone to Open Phil (3.4 co-founder equivalents), capacity building (.8), and overhead (.9). The number of co-founder equivalents focused on GiveWell’s traditional work has stayed low (and relatively constant) since 2012.

Year GiveWell trad-itional research Other output Capacity building Train-ing Open Phil Over-head Total co-founder equiv. Total staff at year-end 2012 0.9 0.4 0.7 0 0.7 0.2 2.9 5.5 2013 0.8 0.6 0.5 0.3 1.4 0.3 3.9 11 2014 0.9 0.4 0.8 0.6 2.6 0.4 5.7 18 2015 0.9 0.7 1.5 1.1 4.1 1.1 9 32

Overall, we find this picture reasonable. While our work on GiveWell’s traditional research has stayed relatively constant in the short term, we think of it as a major benefit of increased staff that we’ve been able to do this while building the Open Philanthropy Project. And we also believe we’ve been laying the groundwork for future capacity increases, since much of our increased staff time has gone into capacity building itself.

Over time, as new staff train and are able to produce more, we hope that this picture begins to shift and that we can devote more capacity to GiveWell’s traditional research, other output, and Open Phil. Our guess is that we will start to see some of this shift in 2016.

Did we make mistakes in allocating staff capacity in 2015?

We see mistakes in our allocation of staff capacity in 2015. Specifically, we didn’t move as quickly as we should have to move staff from one type of work to another. We had several cases where we continued to try to train staff to work on research projects even though we should have already recognized that they were unlikely to be a good long-term fit for thIs type of research, and would be a better fit for another role. It is generally difficult to predict fit, so we classify this as a relatively easy-to-make mistake, though a costly one, since capacity building is a large time investment for relatively senior staff.

Did we make the right choices about how to spend the time that we had for research (i.e., did we choose the best research priorities)?

In addition to staying up to date on our top charities, we prioritized researching the below opportunities with the goal of maximizing our chances of identifying new top charities.

  • Deworming organizations. At the end of 2014, we felt that deworming was one of the most promising interventions we were aware of, but the giving opportunities we had found to scale up deworming had limited room for more funding. In 2015, we prioritized finding new deworming organizations and began evaluations of the END Fund, Sightsavers, and Children Without Worms.
  • Development Media International (DMI). At the end of 2014, we believed there was a significant probability that we would name DMI a top charity at the end of 2015. We prioritized work on DMI with this in mind, but the results from DMI’s randomized controlled trial were not in line with what they would have needed to be in order for DMI to become a top charity.
  • Micronutrient fortification organizations. We prioritized this area because micronutrient fortification programs reach many people at relatively low cost and they are often backed by strong evidence of effectiveness. We prioritized evaluating salt iodization programs run by IGN and GAIN. We also began an evaluation of Project Healthy Children’s programs and invited the Micronutrient Initiative (MI) to apply for a recommendation. MI declined to participate.
  • Bednet organizations. We invited Nothing but Nets to apply for a recommendation (it declined) and we began an evaluation of Episcopal Relief & Development’s NetsforLife® Program.

We believe these priorities were reasonable, given the amount of time we had available and the information we had at the time. However, as these investigations – and further investigations in 2016 – have had fairly disappointing results (in terms of not resulting in new top charities), we are rethinking our approach significantly going forward. More on this in a future post.

Have we failed to publish materials that we should?

We have struggled to publish information about many research projects that we have essentially completed. For example, we have formed views on several interventions for which we have not yet published intervention reports. One major reason for this is that we often try to resolve almost all of our major questions about an intervention before publishing a report on that intervention. A possible solution would be to publish more materials that explicitly note that we are sharing preliminary views that could be substantially affected by additional research. We plan to experiment with publishing more research about preliminary views we’ve reached in the future.

However, a continuing challenge in publishing our research work is that we have limited management capacity to review and sign off on public write-ups. We hope that building management and research capacity will steadily reduce this bottleneck over time.

The post GiveWell’s progress in 2015 appeared first on The GiveWell Blog.

Update on GiveWell’s web traffic / money moved

Fri, 01/08/2016 - 12:34

In addition to evaluations of other charities, GiveWell publishes substantial evaluation of itself, from the quality of its research to its impact on donations. We publish quarterly updates regarding two key metrics: (a) donations to recommended charities and (b) web traffic. This post is being published late due to staff focusing on updating GiveWell’s charity recommendations in the fourth quarter; it also includes a preliminary view of our money moved since the end of our third quarter.

Preliminary estimate of 2015 money moved (since February 1, 2015)

As of early January 2016, we have tracked about $98 million in money moved to our recommended charities. Excluding Good Ventures, we have tracked about $28 million (of which, roughly half has come from donors giving $1 million or more).

These data are preliminary. We expect that in some cases we are currently overstating our impact (e.g. due to double counting or incorrect attribution of our influence) and in other cases we are understating our impact (since there are several weeks left in our metrics year and there are delays entering data); overall, we would guess that we are currently underestimating our annual money moved. We plan to publish our annual metrics (covering February 1, 2015 – January 31, 2016) in March, at which point we will have more confidence in our data and be able to share more details.

GiveWell’s web traffic / money moved through Q3 2015

The tables and chart below present basic information about our growth in money moved and web traffic in the first three quarters of 2015 compared to the previous two years (note 1).

Money moved and donors: first three quarters

Money moved by donors who have never given more than $5,000 in a year increased about 80% to $2.44 million. The total number of donors in the first three quarters increased about 80% to about 8,300 (note 2). These growth rates are reasonably consistent with the growth we previously reported in our first and second quarter metrics.

Web traffic through October 2015

Growth in web traffic excluding Google AdWords increased about 25% in the first three quarters. Last year, we saw a drop in total web traffic because we removed ads on searches that we determined were not driving high quality traffic to our site (i.e. searches with very high bounce rates and very low pages per visit).

GiveWell’s website receives elevated web traffic during “giving season” around December of each year. To adjust for this and emphasize the trend, the chart below shows the rolling sum of unique visitors over the previous twelve months, starting in December 2009 (the first period for which we have 12 months of reliable data due to an issue tracking visits in 2008).

We use web analytics data from two sources: Clicky and Google Analytics (except for those months for which we only have reliable data from one source). The raw data we used to generate the chart and table above (as well as notes on the issues we’ve had and adjustments we’ve made) is in this spreadsheet (note 3, on how we count unique visitors).


Note 1: Since our 2012 annual metrics report we have shifted to a reporting year that starts on February 1, rather than January 1, in order to better capture year-on-year growth in the peak giving months of December and January. Therefore, metrics for the “first three quarters” reported here are for February through October.

Note 2: Our measure of the total number of donors may overestimate the true number. We identify individual donors based on the reported name and email. Donors may not share all of this information or may update it (for example, using a different email), in which case, we may mistakenly treat a donation as if it was made by a new donor. We plan to investigate how large of an overstatement there may be and possibly adjust the total for our next annual metrics report.

Note 3: We count unique visitors over a period as the sum of monthly unique visitors. In other words, if the same person visits the site multiple times in a calendar month, they are counted once. If they visit in multiple months, they are counted once per month.

The post Update on GiveWell’s web traffic / money moved appeared first on The GiveWell Blog.

Update on GiveWell’s web traffic / money moved: Q2 2015

Wed, 09/09/2015 - 15:38

In addition to evaluations of other charities, GiveWell publishes substantial evaluation of itself, from the quality of its research to its impact on donations. We publish quarterly updates regarding two key metrics: (a) donations to top charities and (b) web traffic.

The tables and chart below present basic information about our growth in money moved and web traffic in the first two quarters of 2015 compared to the previous two years (note 1).

Money moved and donors: first two quarters

Money moved by donors who have never given more than $5,000 in a year increased about 80% to $1.54 million. The total number of donors in the first two quarters increased 94% to about 6,000 (note 2).

Most of our money moved is donated near the end of the year (we tracked about 70% of the total in the fourth quarter each of the last two years) and is driven by a relatively small number of large donors. Because of this, we don’t think we can reliably predict our growth and think that our year-to-date total money moved provides relatively limited information about what our year-end money moved is likely to be (note 3). We therefore look at the data above as an indication of growth in our audience.

Web traffic through July 2015

Growth in web traffic excluding Google AdWords increased about 40% in the first two quarters. Last year, we saw a drop in total web traffic because we removed ads on searches that we determined were not driving high quality traffic to our site (i.e. searches with very high bounce rates and very low pages per visit).

GiveWell’s website receives elevated web traffic during “giving season” around December of each year. To adjust for this and emphasize the trend, the chart below shows the rolling sum of unique visitors over the previous twelve months, starting in December 2009 (the first period for which we have 12 months of reliable data due to an issue tracking visits in 2008).

We use web analytics data from two sources: Clicky and Google Analytics (except for those months for which we only have reliable data from one source). The raw data we used to generate the chart and table above (as well as notes on the issues we’ve had and adjustments we’ve made) is in this spreadsheet (note 4, on how we count unique visitors).


Note 1: Since our 2012 annual metrics report we have shifted to a reporting year that starts on February 1, rather than January 1, in order to better capture year-on-year growth in the peak giving months of December and January. Therefore, metrics for the “first two quarters” reported here are for February through July.

Note 2: Our measure of the total number of donors may overestimate the true number. We identify individual donors based on the reported name and email. Donors may not share all of this information or may update it (for example, using a different email), in which case, we may mistakenly treat a donation as if it was made by a new donor. We plan to investigate how large of an overstatement there may be and possibly adjust the total for our next annual metrics report.

Note 3: In total, GiveWell donors directed $5.56 million to our top charities in the first two quarters of this year, compared to $2.41 million that we had tracked in the first two quarters of 2014. For the reason described above, we don’t find this number to be particularly meaningful at this time of year. In the second quarter of 2015, a significant portion our money moved came from one large donation.

Note 4: We count unique visitors over a period as the sum of monthly unique visitors. In other words, if the same person visits the site multiple times in a calendar month, they are counted once. If they visit in multiple months, they are counted once per month.

The post Update on GiveWell’s web traffic / money moved: Q2 2015 appeared first on The GiveWell Blog.

Update on GiveWell’s web traffic / money moved: Q1 2015

Wed, 05/27/2015 - 11:37

In addition to evaluations of other charities, GiveWell publishes substantial evaluation of itself, from the quality of its research to its impact on donations. We publish quarterly updates regarding two key metrics: (a) donations to top charities and (b) web traffic.

The tables and chart below present basic information about our growth in money moved and web traffic in the first quarter of 2015 compared to the last two years (note 1).

Money moved and donors: first quarter

Money moved by donors who have never given more than $5,000 in a year increased 78% to about $760,000. The total number of donors in the first quarter increased to about 3,400. This was up 70% compared to last year, roughly consistent with the last year’s growth.

Most of our money moved is donated near the end of the year (we tracked about 70% of the total in the fourth quarter each of the last two years) and is driven by a relatively small number of large donors. Because of this, our year-to-date total money moved provides relatively limited information, and we don’t think we can reliably predict our year-end money moved (note 2). Mid-year we primarily use data on donations from smaller donors, rather than total money moved, to give a rough indication of how our influence on donations is growing.

Web traffic through April 2015

Growth in web traffic excluding Google AdWords increased moderately in the first quarter. Last year, we saw a drop in total web traffic because we removed ads on searches that we determined were not driving high quality traffic to our site (i.e. searches with very high bounce rates and very low pages per visit).

GiveWell’s website receives elevated web traffic during “giving season” around December of each year. To adjust for this and emphasize the trend, the chart below shows the rolling sum of unique visitors over the previous twelve months, starting in December 2009 (the first period for which we have 12 months of reliable data due to an issue tracking visits in 2008).

We use web analytics data from two sources: Clicky and Google Analytics (except for those months for which we only have reliable data from one source). The data on visitors to our website differs between the two sources. We do not know the cause of discrepancy (though a volunteer with a relevant technical background looked at the data for us to try to find the cause; he didn’t find any obvious problems with the data). (Note on how we count unique visitors.)

The raw data we used to generate the chart and table above (as well as notes on the issues we’ve had and adjustments we’ve made) is in this spreadsheet.


Note 1: Since our 2012 annual metrics report we have shifted to a reporting year that starts on February 1, rather than January 1, in order to better capture year-on-year growth in the peak giving months of December and January. Therefore, metrics for the “first quarter” reported here are for February through April.

Note 2: In total, GiveWell donors directed $1.76 million to our top charities in the first quarter of this year, compared with $1.45 million that we had tracked in the first quarter of 2014. For the reason described above, we don’t find this number to be particularly meaningful at this time of year.

Note 3: We count unique visitors over a period as the sum of monthly unique visitors. In other words, if the same person visits the site multiple times in a calendar month, they are counted once. If they visit in multiple months, they are counted once per month. Google Analytics provides ‘unique visitors by traffic source’ while Clicky provides only ‘visitors by traffic source.’ For that reason, we primarily use Google Analytics data in the calculations to exclude AdWords visitors.

The post Update on GiveWell’s web traffic / money moved: Q1 2015 appeared first on The GiveWell Blog.

GiveWell’s money moved and web traffic in 2014

Mon, 04/13/2015 - 15:34

This is the final post (of six) we have made focused on our self-evaluation and future plans.

This post lays out highlights from our metrics report for 2014. For more detail, see our full metrics report (PDF). Note, we report on “metrics years” that run from February – January; for example, our 2014 data cover February 1, 2014 through January 31, 2015.

  1. In 2014, GiveWell influenced charitable giving in several ways. The following table summarizes the money that were able to track.

  2. In 2014, GiveWell tracked $27.8 million in money moved to our recommended charities, about 60% more than in 2013. This total includes $14.8 million from Good Ventures (up from $9.3 million) and $13.0 million from other donors (up from $8.2 million).

  3. As part of our work on the Open Philanthropy Project, we advised Good Ventures to make grants totaling $8.1 million (this was in addition to Good Ventures’ support for our top charities and standout charities). In addition, the Laura and John Arnold Foundation provided a commitment of up to $6 million to the Meta-Research Innovation Center at Stanford after we connected these organizations.
  4. Our total expenses were $1.8 million in 2014. We estimate that about half supported our traditional top charity work and about half supported the Open Philanthropy Project. Our expenses increased from about $960,000 in 2013 and about $560,000 in 2012 as the size of our staff grew.
  5. Our four top charities received the majority of the $28.1 million tracked to our recommended charities. Our four standout charities received about $1.7m total (mostly from Good Ventures).

  6. In 2014, the number of donors and amount donated increased across each donor size category. Last year, we discussed a substantial decrease among the largest donors from 2012, which we expected might be somewhat temporary. While that category rebounded strongly, it was driven by donors who gave $50,000 or more to our recommended charities for the first time.

  7. In 2014, the total number of donors giving to our recommended charities or to GiveWell unrestricted did not grow significantly (up 9% to about 9,300). This is largely due to many new donors in 2013 (particularly donors who gave less than $1,000) not giving again in 2014.


    Our retention was stronger among donors who gave larger amounts or who first gave to our recommendations prior to 2013. Of larger donors (those who gave $10,000 or more in either of the last two years), about 80% who gave in 2013 gave again in 2014.

  8. Prior to 2013, GiveWell relied on a small number of donors to provide unrestricted support for our operations. Over the last two years, we’ve asked donors for more operational support. In 2014, we raised $3.0 million, up from $1.8 million in 2013 and $0.8 million in 2012. Four institutions and the nine largest individual donors contributed about 75% of GiveWell’s funding in 2014.

  9. We continued to collect information on our donors. We found the picture of our 2014 donors to be broadly consistent with previous information. Based on reports from donors who gave $2,000 or more, we found:
    • The most common ways donors found us was via Peter Singer and personal referrals.
    • Many of the donors are under 40 and work in technology and finance.
  10. Excluding AdWords, unique visitors to our website increased by 9% in 2014 compared to 2013. Including AdWords, unique visitors decreased by 11%. In late 2013, we removed some AdWords campaigns that were driving substantial traffic but appeared to be largely resulting in visitors who were not finding what they were looking for (as evidenced by short visit duration and high bounce rates). Traffic directly to our website increased, but traffic from other non-paid sources was basically unchanged.

  11. In the past, we compared GiveWell’s online money moved to that of Charity Navigator and GuideStar. This year, we did not find data from Charity Navigator and GuideStar so do not have an updated comparison.

For more detail, see our full metrics report (PDF).

The post GiveWell’s money moved and web traffic in 2014 appeared first on The GiveWell Blog.

General progress and plans for GiveWell as an organization in 2015

Tue, 03/17/2015 - 13:20

This is the fifth post (of six) we’re planning to make focused on our self-evaluation and future plans.

Previous posts have discussed our 2014 progress on, and 2015 plans for,

This post outlines our plans and thoughts on issues that cut across these two projects, and pertain to GiveWell the organization as a whole.

Our staff has expanded significantly, and we expect to expand further.

  • At the beginning of 2014, we had 11 full-time staff and 1 Conversation Notes Writer; as of today, we have 18 full-time staff and 8 Conversation Notes Writers.
  • Of our current full-time staff, five work primarily on the Open Philanthropy Project, while the other thirteen do a mix of top charities work and cross-cutting work (including managing Conversation Notes Writers, vetting content from both projects, and administrative work). Currently, our payroll expenses are roughly evenly allocated between the two projects.
  • For most of 2013, Holden and Alexander were the only staff putting much time into the Open Philanthropy Project. In 2014, we shifted more of their time to the Open Philanthropy Project, as well as ~all of Howie Lempel’s time. These additions made it possible for the Open Philanthropy Project to make substantial progress, covered previously. Recently, we’ve added two more full-time staff to the Open Philanthropy Project team.
  • Through 2013, Elie was intimately involved in all charity reviews. He participated in most calls with charity representatives and carefully reviewed the details of all charity-review-related written work. This changed substantially in 2014 with newer staff taking on more research and management responsibility. These additions made it possible for GiveWell to produce much more in 2014 than in 2013, as covered previously. In 2014, we added 5 new staff members who primarily work on top-charities-related work. Natalie Crispin is now the direct supervisor for seven staff members.
  • We are hoping to involve more Research Analysts in the Open Philanthropy Project, particularly for helping with writeups of cause investigations and grants. Because of this, and our desire to build still more capacity for evaluating potential top charities, we are hoping to add 4-8 additional Research Analysts over the next 12 months. There are two future Research Analysts (previously Summer Research Analysts) who have accepted offers and are starting mid-year.
  • In addition, we are starting to seek cause-specific hires for the Open Philanthropy Project (discussed previously), and we have started to advertise for an Outreach Associate position to help us continue to maintain relationships with a growing number of people who give significantly to our top charities.

Improving management was a major focus of 2014. With our larger staff, we have put a lot more attention into recruiting-, training- and retention-related matters. We’ve implemented regular one-on-one meetings, employee satisfaction checkins, and a variety of practices to help staff engage with GiveWell and with each other (regular roundtables on topics of interest, social events, etc.) We’ve also been continually experimenting with moving more responsibilities – including management itself – to employees other than Elie, and as of early 2015 Natalie has taken over management of seven employees. We believe that running an organization of this size has some qualitative differences with running a ~5-person organization, and we’ve put deliberate attention into adapting. We may be writing more about this in the future.

We are planning to launch new websites for both GiveWell and the Open Philanthropy Project this year. We are working with the brand experience firm Cibo on both websites, and are aiming to launch both in April. This is a major time investment for us.

We have several reasons for launching new websites. First, the current GiveWell website was created in 2009 on a low budget, and we think it has major room for improvement. Second, creating separate websites for GiveWell and the Open Philanthropy Project is another step in the direction of creating clear separation between the two (more below). Finally, it is possible we will receive an unusual amount of publicity in 2015: we know of three different books on the subject of effective altruism, and one more that will feature it prominently, all slated to be published in 2015.

We are continuing to work toward separating GiveWell and the Open Philanthropy Project as brands, websites, and – eventually – organizations. In 2014, we struggled with confusion between the two. People we contacted as part of our Open Philanthropy Project work often Googled us, looked at GiveWell, and came away with mistaken impressions about what sorts of giving opportunities we seek for the Open Philanthropy Project. We have been taking small steps to ameliorate this (such as more consistent use of openphilanthropy.org email addresses and signatures), and in 2015 we plan to further work toward separation. In addition to launching a new website for the Open Philanthropy Project – to which we will be moving relevant content currently on GiveWell.org – we will begin conversations about what it would look like to form two separate organizations.

At the same time, we still see substantial overlap between the skills needed for the two projects. If GiveWell gets to the point of having spare capacity, this capacity could be very valuable to the Open Philanthropy Project. Such capacity could include Elie Hassenfeld’s time, if GiveWell becomes less dependent on his involvement; it could also include more junior staff, who could help with creating public writeups on our work among other things. Being able to flexibly allocate employees between the two projects is still valuable for us at this time.

Fundraising remains a priority. We are currently fundraising for unrestricted support, supporting a team that is allocated flexibly between Open Philanthropy Project and our more traditional work. Details are at our December 2014 update. For people donating to GiveWell by webform for regranting to top charities, we have added an option to allocate 10% to GiveWell unrestricted.

Outreach is still not a top priority for us. Our top priority for the Open Philanthropy Project is research (finding outstanding giving opportunities); we feel we have a long way to go on this front before it will make sense to put much effort into outreach. Our top priority for GiveWell is strengthening capacity and making the operation sustainable without needing as much involvement from the co-founders, which could result in a stronger Open Philanthropy Project (see above) as well as a more robust GiveWell.

In light of GiveWell’s maturing research process, and some early signs that growth in money moved from smaller donors is slowing (more in our upcoming metrics report), there is a growing argument for putting more effort into outreach. Still, we see strengthening capacity as the more important goal for the coming year, especially in light of (a) the potential benefits for the Open Philanthropy Project; (b) the fact that there are now substantial efforts outside of GiveWell aiming to drive more donations to our top chariites. The latter include several organizations promoting generous, effective giving as well as the books mentioned above.

We are putting substantial time into new websites this year, which we feel is probably the highest-value activity for outreach. In the future, if GiveWell becomes less dependent on co-founder involvement and higher-capacity, we may further increase our attention to outreach.

We don’t see other major issues (in the “cross-cutting between Open Philanthropy Project and traditional work” category) that need addressing in 2015.

The post General progress and plans for GiveWell as an organization in 2015 appeared first on The GiveWell Blog.

2015 plan for GiveWell’s traditional (“top charities”) work

Thu, 03/05/2015 - 15:06

This is the third post (of six) we’re planning to make focused on our self-evaluation and future plans. The goal of this post is to update GiveWell’s followers on our plans for our traditional work in 2015 and to establish a general set of goals by which we can evaluate ourselves at the beginning of 2016.

As discussed in our previous post, in 2014, GiveWell’s traditional (“top charities”) work conducted a large amount of research while maintaining research quality and building substantially more capacity to conduct research in the future. The amount of money moved to our recommended charities continued to grow; we moved about $28 million to recommended charities in 2014.

This year, our primary goals are to:

  • Build management and research capacity for GiveWell’s traditional work while further reducing senior staff time (note 1) spent on this work, primarily by reallocating Elie Hassenfeld’s management responsibilities related to GiveWell’s traditional work.
  • Maintain our core research product by completing updates on all eight 2014 recommended charities and determining which of them should be recommended as top charities for the 2015 giving season.

Our secondary goals for 2015 are to:

  • Continue to seek outstanding giving opportunities by reviewing 2-4 new charities and publishing 2-4 new intervention reports.
  • Improve the cost-effectiveness analyses and room for more funding analyses in charity reviews.
  • Finish and launch a redesigned GiveWell website.
  • Make further progress on experimental work to “seed” potential recommended charities.

We expect our total output on “top charities” work to be roughly comparable to last year’s, despite a growing staff, because (a) a major focus of the coming year is training, and we expect to trade some short-term efficiency for long-run output; (b) we may be reallocating some capacity from our “top charities” work to the Open Philanthropy Project this year.

More details on some of these goals are below.

Building capacity

In 2015, we hope to build substantially more management and research capacity for GiveWell’s traditional work in order to move toward our goal of having a sustainable organization that is not dependent on past senior staff. As we have discussed before, building capacity is challenging and generally leads to reductions in capacity in the short term. This year, we plan to build capacity by:

  • Training relatively senior staff to take on management roles by reallocating Elie Hassenfeld’s management responsibilities to them as much as possible. For example, Senior Research Analyst Natalie Crispin is currently performing all of Elie’s management responsibilities with respect to GiveWell’s 2015 charity reviews and charity updates. Elie is overseeing Natalie during this transition. Since Natalie is managing others on this work, she does not have as much time to directly do research work herself.
  • Training relatively junior staff to do most charity updates, intervention reports, and new charity reviews. Most junior staff members are relatively new to this type of work.
  • Continuing to hire and train new Research Analysts, Outreach Associates, and Conversation Notes Writers.

We expect that these efforts to build capacity will enable us to do more research – for both GiveWell and the Open Philanthropy Project – in the long run but will reduce the efficiency of our work in the short run, requiring more person-hours per unit of output than in 2014.

Building capacity to do intervention reports

As we wrote in our 2014 self-evaluation post, completing new intervention reports in 2014 was much more difficult and time-consuming than we had anticipated. This year, we are trying to build more capacity for completing these reports by training more staff to do intervention-related research and by improving our process for doing this research. Our ultimate goal is to have a process for completing a reasonable number of intervention reports that does not require substantial involvement from Elie.

We consider building capacity to do more intervention reports to be a high priority because we must be able to complete these reports in order to best prioritize new charities for investigation.

Charity updates

We plan to publish updates on all eight of our recommended charities in 2015. We are generally aiming to have conversations with each charity in February, May, and September that will each result in conversation notes and/or an update report (example). This update schedule may vary somewhat by charity. We are following this charity update schedule so that a) we learn about any major updates that might cause changes in a charity’s recommendation status as soon as possible and b) we spread out the work of refreshing our charity reviews over the course of the year.

As part of our February update calls with recommended charities (especially top charities), we will be gathering more information about charities’ room for more funding situations. As we mentioned in December 2014, we may update our recommended allocation to top charities to reflect any major changes in charities’ funding needs. We tentatively plan to publish such an update in April.

In our charity updates, in addition to our standard questions to follow up on each charity’s activities, we will be focusing in particular on whether any new information from our “standout” charities might lead them to be recommended as “top” charities by the end of the year.

Intervention reports

This year, we hope to complete 2-4 new intervention reports. The programs and program areas that we have preliminarily prioritized for investigation include:

  • Nutrition programs (e.g., folic acid fortification and iron fortification)
  • Immunization programs (e.g., immunization against measles and meningitis)
  • Neglected tropical diseases programs (e.g., trachoma and onchocerciasis)
  • Programs for which we believe a charity would apply if we determined the intervention to be a priority program (e.g., “Targeting the Ultra-Poor” (or “Ultra-Poor Graduation”) programs and voluntary medical male circumcision for the prevention of HIV)

We also plan to publish two intervention reports that are near completion: maternal and neonatal tetanus elimination and mass drug administration to eliminate lymphatic filariasis.

A major factor in prioritizing among intervention reports is determining which interventions seem to be most broadly similar to our other priority programs. We believe that such interventions are most likely to succeed in our current process. Factors that seem to be common among our priority programs include:

  • The program has strong evidence of effectiveness (preferably from multiple high-quality studies)
  • The program is very low-cost per person reached
  • Studies of the program’s effects seem not to be overly dependent on the particular context in which the program was implemented (e.g., studies of health programs often seem more likely to be externally valid than studies of education programs because the mechanisms by which health programs have their effects are often more consistent across populations)
  • The program is highly replicable for an implementing organization (i.e., a charity would face a low burden of proof to show that they were carrying out the same intervention that was studied and shown to be effective)
  • The program has informative proximate outcomes (e.g., deworming pills taken, bed nets delivered and used, etc.) that can be fairly easily measured and monitored

New charity reviews

We plan to actively pursue evaluations of 2-4 new potential top charities this year. Our tentative plans for which charities we may evaluate are below. However, note that there are many reasons that new charity review prioritization could change during the year, such as learning new information from potential recommended charities and completing new intervention reports that change our views on which interventions are promising (several of the new charities that we may evaluate implement interventions for which we have not yet published intervention reports). We also plan to maintain our “open-door policy” for allowing any charity to apply for a recommendation.

As with previous years, we chose to mainly prioritize charities based on our best guess about whether they will become top-rated organizations. We also gave additional weight to organizations that we guessed have some chance of being substantially more cost-effective than our current top charities. The charities that we may evaluate include:

We have also reached out again to Nothing But Nets about applying for a recommendation because it distributes long-lasting insecticide-treated bed nets, which we consider to be one of the most cost-effective priority programs.

Experimental work

In 2015, we (in collaboration with Good Ventures) plan to continue the experimental work to “seed” additional top charities that we began in 2014, though we still do not consider this work to be a high priority. A few activities that we are considering in this area include:

  • Providing funding to promising young charities, such as New Incentives, that could eventually become recommended charities.
  • Funding additional research on and support for scale up for programs that a) could be priority programs if they were supported by additional studies and b) could be scaled up with additional support. The main way in which we are currently doing this is our funding of Evidence Action (early conversation, recent conversation, grant page). We have also investigated other potential partners and may take one or more on in the future.
  • Funding additional independent monitoring that could increase our confidence in the success of recommended charities’ programs (e.g., deworming, salt iodization) or increase our confidence in other organizations’ ability to carry out priority programs (e.g., if we learned that standard government-led bed net distributions were high-quality, then we might recommend additional funding to large non-profits that fund government bed net distributions). We are initially planning to work with IDinsight on this project.

We plan to explore options and publish updates on our progress on this work throughout 2015.

Improving quality of charity reviews

In general, we feel that our charity reviews are high-quality. However, we believe that there are some ways in which they could be improved. In particular, in 2015, we hope to improve the cost-effectiveness analyses (CEs) (example) and “room for more funding” analyses (example) in our charity reviews, if we have the capacity to do so.

We feel that the quality of our CEs has been acceptable in the past, but we have identified tangible ways to improve them and feel that it is worth using some of our additional capacity to do this because such analyses are relatively important to our charity recommendations. In particular, we aim to:

  • Generally improve the transparency and clarity of our CEs.
  • Think more carefully about the major inputs that cause a substantial amount of variation in our CEs and ensure that we know as much as we can about those inputs. For example, the proportion of deworming pills that were given to children as part of Schistosomiasis Control Initiative (SCI)‘s campaigns is a relatively important parameter in our CE for SCI, but we did not have as much confidence in our understanding of this parameter as we could have at the end of last year.
  • Ensure that we properly account for “leverage” considerations when appropriate (e.g., in our CEs for organizations such as Iodine Global Network and Deworm the World Initiative).
  • Ensure that we are applying rules for including and excluding costs and benefits consistently across all CEs (e.g., making sure that we have captured all in-kind donations to all of our charities’ programs in our CEs).

Similarly, we would like to improve the “room for more funding” sections of our charity reviews. To achieve this, we plan to:

  • Discuss charities’ funding needs with them earlier in the year so that we can gain as much clarity as possible about their funding situations.
  • Standardize the questions that we ask charities about their room for more funding so that we can be more confident that we are making similar comparisons across organizations.

How large is the pipeline of potential top charities and priority programs?We are building a charity review organization with a substantial amount of capacity for conducting research, but it is unclear how many charities that we have not yet reviewed may be competitive with our current top charities.

One factor that affects our estimate of the size of the pipeline is that, over time, we have broadened our criteria and research process so that we are able to evaluate more types of potentially high-impact giving opportunities. For example, in the last two years we have begun to evaluate organizations that play an advocacy and advisory role to governments, such as Deworm the World Initiative and Iodine Global Network. We are also now open to evaluating components of “mega-charities” that are working to scale up potential priority programs, such as UNICEF Maternal and Neonatal Tetanus Elimination Initiative.

We have also seen that charities have been more willing to engage in our review process over time. This may be due to our growing money moved, growing influence, and generally improved incentives for charities to apply (discussed in our 2014 self-evaluation post).

These changes have had the effect of increasing the pipeline of potential charities and programs to evaluate and potentially allowing us to find more cost-effective giving opportunities than we had been able to find previously.

We are also trying to increase the pipeline of outstanding giving opportunities through our experimental work mentioned above (e.g., by funding New Incentives and replications of studies).

Currently, we do not have a strong sense of the overall number of potential top charities and potential priority programs that we have not yet evaluated, but we feel that there are substantially more promising charities and programs than we will be able to evaluate this year and believe that it is possible that the pipeline will grow over time.

Note 1: In this post, senior staff refers to Elie, Holden, and Alexander. Many staff took on additional responsibilities throughout 2014, so this refers to senior staff as of January 2014, not as of today.

The post 2015 plan for GiveWell’s traditional (“top charities”) work appeared first on The GiveWell Blog.