Recommendations: K-12 education (focus on NYC)

  • Research shows drastic differences in academic performance - and later life outcomes - between those who grow up in low-income households and others, as well as between ethnic minorities and whites.1
  • This gap exists before children enter school, appearing as early as age 5.2
  • We recommend the Knowledge is Power Program (KIPP).
  • We also recommend that donors interested in the cause of improving equality in the United States more broadly, consider supporting the cause of early childhood care for which stronger evidence of effectiveness exists.

Table of Contents

The organizations in this area are working on a difficult problem, seeking to address a gap that is significant and deep-rooted (in fact it is a gap that is visible by the age of five). Most of the organizations we looked at were unwilling or unable to provide rigorous empirical evidence that they have improved academic outcomes for disadvantaged children; two organizations, the Knowledge is Power Program and Teach For America, made convincing cases, and we recommend them to donors. A more detailed overview of our reasoning is available at our in-depth discussion of K-12 education.

Recommendations

Our recommended organizations are below; if you would like to donate to one, you can do so using the appropriate "Donate now" link. GiveWell has no involvement in the donation process, and collects no fees on it.

  1. Knowledge is Power Program. KIPP is a network of more than 50 charter schools (mostly grades 5-8) across the U.S., including 4 in New York City, focusing on improving academic outcomes for disadvantaged youth. All students in a school's vicinity are eligible for admission, which is determined by random lottery, and tuition is free. KIPP pays for its operations through a combination of funds from the local public school system and donations; we estimate that KIPP spends $1,000-5,000 per student (excluding per-pupil funding KIPP receives directly from the government, which accounts for an additional $10,000-$15,000 per student).

    Two quasi-experimental studies and our own analysis of all publicly available test data give us moderate confidence that KIPP is improving its students' academic performance in a significant and measurable way (one that shows up in test scores), and that it is doing so consistently across its large national network of schools. Donate now through Google checkout

  2. Teach For America. TFA recruits, trains and places recent college graduates in low-income public schools across the country; the aim is both to provide quality teachers in the short term and to recruit talented people into the education sector in the long term. (Note that the goal of drawing talented people into the education sector is implicitly a goal of recruiting them away from other pursuits; whether this goal is a net positive or net negative to society more broadly is an open question.)

    We have no substantial information on TFA's long term impact, other than anecdotes of particular TFA graduates who have made contributions to the field of education. There is a compelling research case, however, regarding the short-term goal: TFA teachers perform at least as well as - and in some cases slightly better than - the teachers they are replacing, in terms of student improvement on standardized test scores. Donate now through Network for Good

For donors interested in K-12 Education, we recommend the Knowledge is Power Program and Teach For America. This page outlines why.

The achievement gap

Our overview of the achievement gap discusses the large discrepancy between different groups in the U.S. - both socioeconomic (e.g., high- vs. low-income families) and ethnic (e.g., African-American vs. white). The same groups that score lower on measures of academic achievement also have lower incomes and even higher incarceration rates, a phenomenon that has made some conjecture that education is a key to later-life success. The charities we examined for this cause aim to improve academic outcomes for disadvantaged youth (generally low-income members of underperforming ethnic groups).

Yet it's important to recognize that this achievement gap is already present before children enter kindergarten, and grows only slightly afterward. Any organization focused on K-12 education, therefore, is likely trying to help children who are already struggling - and simply equalizing the quality of education (for underperforming and overperforming groups) would not necessarily close the achievement gap between them.

We therefore feel it is appropriate to approach the cause of K-12 education with skepticism. We cannot assume that a program is improving children's academic performance simply because it is providing them with extra support or moving them to better schools; instead, we seek a program with a compelling empirical case.

The challenge of measuring success

Ideally, we would consider not just a charity's effect on students' academic performance, but its long-term effect on their lives. Much of the research on early childhood care attempts to gauge the impact of care on a variety of later life outcomes, including clients' mental health and their likelihood of having a criminal record; however, we believe that very little similar research exists on the impact of K-12 education. (We have seen a few studies on the correlation between academic performance and later earnings, but academic performance can reflect pre-existing abilities; we have seen no studies attempting to isolate the effect of education on earnings.)

In the absence of data on later life outcomes, we have chosen to focus - for this cause - on charities' abilities to improve their clients' academic performance. This choice meshes with the way applicants chose to make their case, in our open-ended application; applicants commonly sent information about their applicants' high school graduation rates and achievement test scores (and less commonly about other academic-related measures, such as attendance rates and college enrollment rates).

In our view, however, it is not enough to demonstrate that a program's participants outperform non-participants (as many of our applicants did). In evaluating education-related data, we believe that selection bias is a major issue: the fact that one student enters a program (charter school, summer school, tutoring session, etc.) while another does not may indicate pre-existing differences in ability and/or motivation that cause the students both to act differently (i.e., one enters the program and one does not) and to perform differently (i.e., one has more academic success than the other), regardless of the effects of the program itself. One good example of this phenomenon is seen in the research on the federal Talent Search program: program participants outperformed non-participants in many ways (including test scores) that could not be plausibly attributed to the program.

Our strongest applicants demonstrate an impact on student performance that is difficult to attribute to pre-existing differences between participants and non-participants.

Rigorously evaluated programs

Both the Children's Scholarship Fund and Teach For America run programs that have been evaluated through randomized controlled trials. (A good and brief explanation of what a randomized controlled trial is and why it provides particularly strong evidence is available via the Poverty Action Lab). KIPP, a national network of charter schools, does not currently have similar studies available, but enough data exists on its students' performance over time that we were able to make an educated guess about its effectiveness, even with concerns about selection bias taken into account. Below we summarize the evidential case for these three organizations' effectiveness; see the full reviews of the organizations for more detail.

Children's Scholarship Fund (CSF)

The Children's Scholarship Fund offers partial-tuition scholarships to low-income children, so that they can attend private rather than public school. We find this to be an intuitively appealing intervention, since it uses a fairly simple mechanism to allow families to make drastic and long-lasting changes in the type of education their children are receiving (and in particular to get them out of the worst schools). However, as we explain in detail in our review of the Children's Scholarship Fund, the empirical evidence suggests that such a program's overall effects on participants' academic performance are negligible (i.e., when scholarships have been offered to some students and not others, via lottery, those offered scholarships do not appear to have had better academic performance). This may be because partial-tuition scholarships do not change many families' behavior (i.e., the families willing to pay partial tuition for private school would also be willing to pay full tuition, and thus the scholarship ultimately has no impact on what school their child attends); it may be because the differences between public and private schools are not sufficient to overcome substantial and deep-rooted disadvantages; or it may be for another reason. In any case, we feel that the available evidence does not support this intuitively appealing program. See our full review for more.

Teach For America (TFA)

Teach For America recruits, trains and places recent college graduates in low-income public schools across the country; the aim is both to provide quality teachers in the short term and to recruit talented people into the education sector in the long term. A reasonably large body of evidence indicates that students assigned to TFA teachers' classrooms (in some cases randomly) perform slightly better than similar students assigned to traditional students' classrooms. The effect is small, but statistically significant. Ignoring the potential benefits (and costs) of recruiting more people into the education sector, we feel that TFA can be classified as a proven, effective, scalable program that is contributing to better academic outcomes. See our full review for more.

Knowledge is Power Program (KIPP)

KIPP is a network of more than 50 charter schools (mostly grades 5-8) across the country, including 4 in New York City, focusing on improving outcomes for disadvantaged youth. Unlike with CSF and TFA, there is currently no evidence available about KIPP's effectiveness from randomized controlled trials. However, by examining trends over time in the performance of its students, we concluded that its students see substantially better improvements than students of nearby schools - and that it is unlikely (though not impossible) that this difference can be attributed to selection bias. (The heart of our analysis is the fact that KIPP's students are generally not performing much better than others at the time they enter KIPP, but within a year of entering are substantially outperforming others; this makes it somewhat difficult to imagine that their greater success can be attributed to pre-existing, non-KIPP-related qualities). This evidence is not as rigorous as the evidence for TFA (i.e., TFA's evaluations leave less room for doubt), but qualitatively speaking, KIPP's impact appears more substantial than TFA's. See our full review for more.

Ultimately, we feel that both KIPP and TFA are running proven, effective, scalable programs that improve academic outcomes for their students; that these qualities are impressive and rare; and that deciding between the two is largely a judgment call. We recommend either to a donor interested in this cause. We recommend KIPP before TFA because we feel that its model more effectively concentrates the benefits of improved education: a KIPP student enters a substantially different educational environment for years, and is thus more likely to experience a significant change in performance and truly overcome the substantial disadvantages that lower-income children face. By contrast, TFA's benefits appear largely "spread out," i.e., small benefits for a large number of students.

Other applicants

We invited a total of 113 organizations to apply within this cause. (Details of how we found applicants.) 50 completed our Round 1 application; 54 did not apply (the remaining 9 applied in other causes). The goal of our Round 1 application was to identify the charities most likely to be able to demonstrate proven past success in improving educational outcomes for disadvantaged students. Details on the criteria we used to choose finalists - along with the applications of all who applied, finalists and non-finalists - are available here.

We named 12 finalists, all of which provided at least some evidence on the performance of their clients vs. non-clients that we felt was worth a second look (either because it showed very large differences, or because we had reasons to believe that the selection bias issues discussed above might not be a problem). After carefully reviewing all 12 second-round applications, we concluded that 3 of our applicants (the ones discussed directly above) had provided sufficient evidence to assess their effectiveness. For the other 9, we concluded that there were too many questions about the available data (most of them pertaining to selection bias concerns, or simply to small and unrepresentative data sets) to conclude that their programs have empirically demonstrated effectiveness.

List of other finalists

Achievement First

What they do: Achievement First is a network of 12 charter schools serving 2500 students in Connecticut (New Haven and Bridgeport) and Brooklyn (Attachment A-2 Pg 1). (3 more schools will be opening during fall of 2008.)

Available evidence: Achievement First sent us achievement test score data from selected classes, showing large gains for 5th grade students and higher overall scores than students in nearby public schools (Attachment A-2 Pgs 9-10 and Attachment A-1 Pgs 26-30).

Why we don't recommend this organization: We aren't confident that the results Achievement First submitted are representative of the organization as a whole. We were sent data only for selected classes, and were not able to obtain much more data independently (as we were for KIPP) or to locate studies specifically addressing concerns about selection bias (as we did for KIPP).

Double Discovery Center

Double Discovery Center requested complete confidentiality for all of its application materials. We are therefore unable to give the full details of our reasoning. Overall, we found Double Discovery Center to be similar to other applicants in this section, in that it provided data that we felt was insufficient due to issues along the lines of those discussed above.

Harlem Center for Education

What they do: Harlem Center for Education is a community organization running programs focused on low-income youth and adults in Harlem, including the federally funded (and evaluated) Talent Search Program, which aims to increase the likelihood that low-income students attend college through, among other things, college counseling, SAT preparation, and tutoring (Attachment A-1 Pg 1).

Available evidence: As our overview of the federal Talent Search program states, we are somewhat optimistic about the program's general approach, but not enough to support an organization simply because the organization is affiliated with this program. HCE did submit a comparison of college enrollment rates for its students relative to the average enrollment for low-income students in New York State (Attachment B-2 Pg 6).

Why we don't recommend this organization: Insufficient empirical case due to concerns over selection bias. We find it likely that a student's choice to participate in – and complete – the Talent Search program is reflective of other relevant characteristics which would lead to higher graduation rates, a concern that is born out by existing evidence about the program at the federal level.

Learning through an Expanded Arts Program (LEAP)

What they do: LEAP runs a variety of programs; its application focused on the ALLL (Active Learning Leads to Literacy) program, 100 hours (20 weeks, 1 hr per day) of instruction in arts and other non-academic subjects to students in grades K-2 (Attachment B-2 Pg 2). The program includes in-class instruction and field-trips and is administered jointly between the students' teacher (for 3 days out of the week) and a LEAP instructor (for 2 days of the week) (Attachment B-2 Pg 2).

Available evidence:In a pilot study of ALLL including 5,337 students at 25 schools, students randomly assigned to ALLL classrooms showed better yearly gains on the Early Childhood Language Assessment System (ECLAS) tests than did students randomly assigned to non-ALLL control classes (Attachment B-2 Pgs 2-4).

Why we don't recommend this organization: The ALLL program only constitutes about 13% (Attachment A-1 Pg 4) of LEAP's total expected budget for 2007-08 (Attachment D-2); we do not have sufficient empirical evidence about LEAP's wide variety of other activities.

Leadership Enterprise for a Diverse America (LEDA)

What they do: LEDA holds a seven-week program at Princeton University, targeting students from minority (African-American, Latino, and Native American) backgrounds (Attachment A-2 Pg 1) and relatively low-income families (Attachment B-9). The program consists largely of college-style academic work that aims to prepare students academically and socially for college (Attachment A-2 Pg 3). In addition, the program offers student-specific guidance to help scholars prepare for the college application process (Attachment A-2 Pg 3).

Available evidence: In a study of participants in the first year of the program, LEDA found that its scholars are more likely to enroll in selective colleges than those who were finalists but ultimately not selected for the program (Attachment B-9).

Why we don't recommend this organization: We found that LEDA scholars and non-scholar finalists cannot safely be compared. LEDA scholars are significantly more academically proficient than finalists who aren't selected (see this table; data comes from Attachment B-9), leading us to believe that the difference between scholars' and finalists' performance may be a reflection of their abilities rather than any impact of the program.

New Visions for Public Schools

What they do: New Visions runs a wide variety of programs aiming to improve NYC's public schools. Its New Century High Schools (NCHS) Initiative replaced large high schools with smaller schools; its other programs focus largely on training, information sharing, and other resources for teachers and school leaders (see Attachment A-3 for a full list).

Available evidence: A third-year report on the NCHS Initiative compared students at NCHS small schools to students at comparison schools who were otherwise similar, as measured by characteristics including ethnicity, eligibility for free- and reduced-price lunch, and previous achievement test scores; NCHS students had higher attendance rates and grade promotion rates, but lower scores on Regents exams (Attachment B-1 Pg 39-46). A fourth-year report found similar results (higher graduation rates, fewer students graduating with Regents and Advanced Regents diplomas) using a similar though not identical methodology (see Attachment B-24 Pg 39-41).

Why we don't recommend this organization: Our concerns about the NCHS study include (a) the fact that students appeared to perform better on measures that are directly controlled by the school (e.g., grade promotion and graduation) but not on more direct measures of performance (e.g., performance on Regents exams); (b) the fact that while students were matched on some observable characteristics, this technique does not fully account for potential selection bias in the students who choose to attend NCHS schools (and in particular - unlike the analysis we did for KIPP - does not show any measure by which NCHS and non-NCHS students started at comparable levels and improved at different rates); (c) evaluations of ongoing small schools projects do not provide evidence of an effect on academic achievement. (See for example, this report on the Federal Smaller Learning Communities Program, starting at Pg 115.)

In addition, New Visions's focus now appears to be on other programs, particularly teacher and principal training programs, for which we have not seen an empirical case.

Replications, Inc.

What they do: Replications, Inc. identifies high-performing public middle schools (6th-8th grades) and high schools and "replicates" them as New York City public schools. The process includes placing the school's future principal in the "model school" for training. Replications, Inc. has started 24 schools in New York City (Attachment A-1 Pg 1).

Available evidence: Achievement test data shows that Replications students outperform students in nearby public schools (see this chart of test scores). (The data we used is available in this excel file.)

Why we don't recommend this organization: Insufficient empirical case, due to concerns over selection bias with the data we do have. In the data we've seen, Replications students outperform students in nearby public schools from year one, and the gap does not appear to grow, suggesting that the difference may be one of students rather than one of schooling.

St. Aloysius

What they do: St. Aloysius runs a school (pre-K through 8th grade) serving 300 students in Harlem (Attachment A-2 Pg 1). Its program includes a mandatory summer academy for 5th-7th graders and after-school activities that are attended by the vast majority of students in all grades (Attachment A-2 Pg 3).

Available evidence: St. Aloysius students perform significantly better than students in district public schools on the New York State English Language Arts exam, although their performance on math exams is mixed (Attachment A-2 Pg 6). St. Aloysius students have also graduated at a rate well above the city average (98% St. Aloysius over the last 15 years (Attachment A-1 Pgs 6-7) - compared to 43% for NYC as a whole, according to a Manhattan Institute study).

Why we don't recommend this organization: Insufficient empirical case, due to concerns over selection bias with the data we do have. St. Aloysius students pay an average of $2,400 in tuition (Attachment A-2 Pg 5); the willingness to do so may indicate a combination of family income, family environment, motivation, etc. that makes it inappropriate to compare St. Aloysius students to NYC students in general, and difficult to put their performance in context.

Student Sponsor Partners

What they do: Student Sponsor Partners funds approximately 400 high-school scholarships for disadvantaged rising ninth-graders, enabling them to attend one of 22 non-public "Partner Schools" in New York City (Attachment A-1 Pg 1). SSP also provides each scholarship recipient with a Sponsor who meets with the student 4-6 times per year and pays a portion of the student's tuition (Attachment A-1 Pg 2).

Available evidence: SSP provided data illustrating that its students graduate high school and matriculate to college at rates that are 2-3 times those of the public schools in which SSP students would otherwise have been enrolled (based on SSP's range of graduation rates in their feeder schools found in Attachment A-1 Pg 1), and almost twice the rate for their demographic groups in NYC (Attachment A-2 Pg 4 and Attachment A-1 Pg 2).

Why we don't recommend this organization: Insufficient empirical case, due to concerns over selection bias with the data we do have. We feel that the feeder school graduation rate does not provide a reasonable comparison group, given that SSP's participants enter the program performing better than students attending feeder schools. SSP students are more likely to be at Level 4 (the highest performing level) and less likely to score at Level 1 (the lowest performing level) even before they enter the program (Attachment A-3 pgs 2 and 7).