GiveWell aims to find the best giving opportunities we can and recommend them to donors. We tend to put a lot of investigation into the organizations we find most promising, and de-prioritize others based on limited information. When we decide not to prioritize an organization, we try to create a brief writeup of our thoughts on that charity because we want to be as transparent as possible about our reasoning.
The following write-up should be viewed in this context: it explains why we determined that (for the time being), we won't be prioritizing the organization in question as potential top charity. This write-up should not be taken as a "negative rating" of the charities. Rather, it is our attempt to be as clear as possible about the process by which we came to our top recommendations.
The last time we examined the charities working primarily in the U.S. was in 2010. As of 2011, we have de-prioritized further work on this cause.
The content we created in 2010 appears below. This content is likely to be no longer fully accurate, both with respect to what it says about the organization and with respect to what it implies about our own views and positions.
Big Brothers Big Sisters of America is centered around one-on-one mentoring relationships between adults and children aged 4-19 years old.1 Big Brothers Big Sisters serves approximately 245,000 children nationwide.2
The child-mentor outings are "a few hours, a couple times a month."3 Mentors are intended to participate in a variety of activites with children such as playing sports, hiking, reading books, or just spending time together.4
Big Brothers Big Sisters of America's two core programs are community-based and school-based mentoring.5 In the community-based program, children and mentors meet on weekends or in the evenings.6 In the school based programs, the child and the mentor meet in school, inside or outside the classroom.7
Big Brothers Big Sisters also runs five special mentoring programs for specific demographics: African-American Mentoring,8 Hispanic Mentoring,9 Native American Mentoring,10 Mentoring Military Children,11 and the Amachi Program (for children with an incarcerated parent).12
Big Brothers Big Sisters of America is distinct from the many, local chapters of Big Brothers Big Sisters (e.g, Big Brothers Big Sisters of New York City). We are not currently clear on the specific role Big Brothers Big Sisters of America plays.
Compared to other charities in this general area, Big Brothers Big Sisters has a relatively large body of rigorous research behind its programs. Big Brothers Big Sisters has commissioned randomized controlled trials of both its core programs, and a Campbell Collaboration literature review is cautiously optimistic about the impact of mentoring programs. We note that all of the measured impacts are short-term in nature: the community program only assessed results immediately after the program's end, and only one of the effects identified by the school program were statistically significant13 by the second year evaluation.
The Big Brothers Big Sisters community mentoring Program was the subject of a randomized controlled trial reviewed by Coalition for Evidence-Based Policy.14 1,138 youths were included in the study, which evaluated the program over an 18-month period (October 1991-February 1993).15 Effects were measured at the end of the program,16 finding the following statistically significant (at p
In the evaluation, mentors frequently met with mentees: "Little Brothers and Little Sisters met with their Big Brothers and Big Sisters on a regular basis. More than 70% of the youths met with their Big Brother or Sister at least three times a month, and approximately 45% met one or more times per week. An average meeting lasted 3.6 hours."20
The Big Brothers Big Sisters school mentoring program was the subject of a randomized controlled trial.21 The study evaluated programs at 10 sites across the country.22 1,139 students in grades 4-9 were included in the evaluation.23 Data was collected by surveying youth, mentors, and school officials.24
After one year, the study found statistically significant effects on 5 of 23 reported school-related outcomes (such as self-reported skipping school or teacher reported behavior.25 The study found no statistically significant impacts on 8 non-school related outcomes.26
At the second-year follow up, the only statistically significant effect (at p27 The non-school related outcomes remained non-statistically significant.28 Note that only 52% of mentees retained a mentor in year 2.29
We have reviewed Big Brothers Big Sisters of America's website for information regarding its ongoing monitoring of its programs to assess whether results are in line with those found in the studies discussed above.
Big Brothers Big Sisters reports on its website that it tracks ongoing data on participants' academics, relationships, attitudes and behaviors.31 We did not find output from this tool on the website, and contacted Big Brothers Big Sisters to discuss this. We contacted Big Brothers Big Sisters about this issue in October; despite some back-and-forth the organization has not yet been available for a conversation.
Big Brothers Big Sisters recently commissioned a survey of adults who had participated in its programs as children.32 The survey attempted to match Big Brothers Big Sisters alumni with a control group who were demographically similar as children.33
We have only seen summaries of the results34 and not technical reports containing such as detailed information on selection methodology or direct comparisons of childhood demographics for alumni and controls. We therefore don't believe that donors should put significant weight on the generally positive results, but we do commend Big Brothers Big Sisters for its rare (among the charities we've reviewed) attempt to assess long-term outcomes for participants of its programs.
We do not have data from Big Brothers Big Sisters of America that would allow us to calculate a cost per student served. Nevertheless, we give a very rough estimate of $1,000 per child served for its community and school based mentoring programs. This is based on:
Age of children served:
Big Brothers Big Sisters of America, "Annual Report 2009," Pg 3.
"With programs located in communities across the country, the Big Brothers Big Sisters network last year served 245,000 children in need of positive role models. Those children were mentored by caring volunteers through our two core programs, community-based and school-based mentoring." Big Brothers Big Sisters of America, "Annual Report 2009," Pg 3.
"Being a Big Brother or Big Sister is one of the most enjoyable things you'll ever do.... You and your Little can share the kinds of activities you already like to do. Play sports together. Go on a hike. Read books. Eat a pizza with extra anchovies. Or just give some advice and inspiration." Big Brothers Big Sisters of America, "Volunteering Is Fun, Big Time."
Big Brothers Big Sisters of America, "Our Programs Get Things Started."
"Some Bigs meet their Littles on the weekends. Others get together with their Littles in the evenings. Each match is unique and develops a schedule that works for them." Big Brothers Big Sisters of America, "Community-Based Mentoring."
"It may sound obvious, but sometimes the best place to help a child realize their potential is at school. Kids enjoy having their Bigs meet with them there, whether it's in the classroom or on the playground. And parents know what a positive impact it makes." Big Brothers Big Sisters of America, "School-Based Mentoring."
Big Brothers Big Sisters of America, "African-American Mentoring."
Big Brothers Big Sisters of America, "Hispanic Mentoring."
Big Brothers Big Sisters of America, "Native American Mentoring."
Big Brothers Big Sisters of America, "Military Mentoring."
Big Brothers Big Sisters of America, "Amachi Program."
Throughout this page, by 'statistically significant' we mean significant at 5% significance level.
Coalition for Evidence-Based Policy, "Big Brothers Big Sisters."
"This program was evaluated in one randomized controlled trial of all 1,138 youths, age 10-16, who applied to one of eight large Big Brothers Big Sisters agencies in various U.S. cities between October 1991 and February 1993, met the program's eligibility requirements, and agreed to participate in the study." Coalition for Evidence-Based Policy, "Big Brothers Big Sisters."
"Effect of Big Brothers Big Sisters on youths in the intervention group, 18 months after random assignment (compared to the control group)." Coalition for Evidence-Based Policy, "Big Brothers Big Sisters."
“We found, as shown in Table 6, that Little Brothers and Little Sisters were statistically significantly less likely than their control counterparts to start using illegal drugs and alcohol during the study period. During the 18-month follow-up period, 11.47% of the control youths started using drugs. Little Brothers and Little Sisters, on the other hand, were 45.8% less likely to start using illegal drugs than were their control counterparts.” Grossman and Tierney 1998, Pg 413; see also Table 6, Pg 414.
“Big Brothers and Big Sisters also had an effect on youth's hitting behavior. On average, the number of times Little Brothers and Little Sisters reported hitting others during the previous 12 months was 32% less than that of the control youths.” Grossman and Tierney 1998, Pg 413; see also Table 6, Pg 414.
“Underlying the improvement in grades, we found improvement in school attendance. At the end of the study period, Little Brothers and Little Sisters had skipped 52% fewer days.” Grossman and Tierney 1998, Pg 415; see also Table 7, Pg 416.
Grossman and Tierney 1998, Pg 410.
"Youth were recruited into the SBM programs as they normally are—mostly through referrals by school staff—during the spring prior to data collection as well as the fall of the first year of the study (2004). Youth who were accepted into the program were assigned randomly to either a treatment or control group. Those assigned to the treatment group were matched in the usual manner with a volunteer mentor. Youth in the control group were placed on the agency's waiting list for the duration of the study period. All youngsters assigned to these groups were followed throughout the study and included in all impact analyses." Herrera et al. 2007, Pg 5.
"The ten BBBS agencies selected for this study differed in size and geographical location (see Table 1)." Herrera et al. 2007, Pg 6.
"To participate in the study, youth were required to be fourth through ninth graders (9 to 16 years old) at the start of the study, to have parental consent, and to not have been referred to the program due to a crisis (for example, youth referred from Child Protective Services). A total of 1,140 youth attending 71 schools met these criteria. Permission was later withdrawn for one child in the control group, leaving a total of 1,139 study participants: 565 youth who were randomly assigned to the treatment group and 574 assigned to the control group." Herrera et al. 2007, Pg 6.
"We attempted to collect data for all members of both the treatment and control groups through surveys administered to teachers and youth and (for the treatment group) mentors at each of three time points: the beginning of the 2004-2005 school year (the baseline); in spring of the 2004-2005 school year (first follow-up); and in late Fall 2005, 15 months after the start of the baseline (second follow-up). Mentors also completed one additional survey in early Fall 2005, which provided information on summer communication with their Littles. Teacher and youth surveys measured a wide range of academic, behavioral and social outcomes; both youth and mentors reported on relationship quality; and mentors provided information on match inter-actions, summer meetings and program characteristics. Mentors whose matches ended prior to the first or second follow-up completed a match closure form that asked why the match had terminated. In addition to the quantitative data collection, we conducted agency interviews at the beginning of the study, focusing on the agency's history and infrastructure. We also conducted visits to each of the 10 agencies and two participating schools at each agency in Spring 2005. During these visits, we interviewed agency staff as well as teachers, principals and school liaisons (i.e., school staff, typically a counselor or principal, responsible for coordinating the program with BBBS staff) to learn about school characteristics, agency relationships with the schools, program staffing and perceptions of program benefits and drawbacks." Herrera et al. 2007, Pg 7.
Herrera et al. 2007, Pgs 34-5.
Herrera et al. 2007, Pg 38.
Herrera et al. 2007, Pg 48.
"In keeping with the results we found at the end of the first school year, no overall impacts on out-of-school outcomes were significant 15 months into the study (see Table 17 on the next page). Littles reported levels of parental and peer support, self-esteem, substance use and other types of misbehavior that were similar to their non-mentored peers." Herrera et al. 2007, Pg 49.
"At the time of the second follow-up (in late fall of the second school year), only 52 percent of the Littles were receiving mentoring. In all, 41 percent of the Littles were meeting with the mentor they had met with in the previous school year, and another 11 percent were meeting with a new mentor. (Only 18 percent of Littles met with a mentor for all three school semesters in the study.) A few Littles (2.5 percent) met for some period of time with a mentor during the second school year but were no longer meeting by the second follow-up.55In many cases, the first-year match ended because the Little transferred to a new school, often as a result of the normal transition from fifth to sixth grade. The youth participating in the study (both the Littles and their non-mentored peers) attended 71 schools at baseline but spanned close to 300 by the end of the study. Close to one third (32 per-cent) of Littles had changed schools by the second follow-up,56 and fewer than half (46 percent) of these transferring Littles continued to receive mentoring from the BBBS SBM program in their new schools."
Herrera et al. 2007, Pg 46.
Tolan et al. 2008, Pg 2.
"Big Brothers Big Sisters of America implemented a tool called the Youth Outcomes Survey in 2009, powered by our groundbreaking technology system, the Agency Information Management System (AIM). AIM is a unique and proprietary database that allows us to track and evaluate relationships between our volunteers and the children we serve. Our new outcomes measure tracks annual progress of children in our program in three key areas:
Survey results will be closely monitored and help us better meet the needs of each individual child."
Big Brothers Big Sisters of America, "Annual Report (2009)," Pg 7.
"To better understand the longer-term value of Big Brothers Big Sisters programs, we commissioned Harris Interactive to contact a group of adults who had been Littles for over a year to ask questions about their experience as Littles and about their wellbeing as adults. The new data gives us a glimpse as to what happens when Littles become adults.Between March 3 and April 16, 2009, Harris Interactive conducted an online survey of 200 adults who participated in Big Brothers Big Sisters as Littles for at least one year during their childhood. There were 109 alumni sampled from Harris Interactive's panel of double opt-in online respondents, called the Harris Poll OnlineSM panel. In addition, alumni were sampled from lists provided by Big Brothers Big Sisters consisting of names provided by 119 of our agencies across the country, as well as people who self-identified as alumni through our e-newsletter subscriber list or the Big Brothers Big Sisters of America website." Big Brothers Big Sisters of America, "Annual Report (2009)," Pg 15.
"This evaluation did not include a cost study, so we cannot precisely document the annual cost of supporting an additional match. Based on the annual budgets of the eight study agencies and their staffing patterns, however, $1,000 seems a reasonable estimate of the cost of making and supporting each additional
match." Grossman and Tierney 2000, Pgs 31-32.
"The average SBM budget was $798,790 for all programs but only $569,357 when excluding the outlying program (see Table 19). One of the major reasons the budgets varied as widely as they did was that the programs served very different numbers of children. The multimillion-dollar agency served more than 2,000 children in its SBM program, while the smallest agency served 155. The average number of children the agencies served in their SBM programs was 781, or 643 without the largest program. To account for the variation in the number of children served, we examined costs per youth. Across the 10 school-based programs, the cost ranged from $370 to $1,415 per youth per year. The average annual cost was $987, while the median was $1,064." Herrera et al. 2007, Pg 59.