Note: consistent with our usual practices as well as our agreement with the Pew Charitable Trusts (Pew), Pew reviewed a draft of this page prior to publication but the final product is ours.

Good Ventures with input from GiveWell, awarded a grant of $3 million over two years to the Pew Public Safety Performance Project (PSPP) as part of an investigation of criminal justice reform. Throughout the rest of this page, "we" refers to both Good Ventures and GiveWell, as the two organizations are closely collaborating on this work. We plan on regularly following up on this grant and also expect PSPP to be a helpful collaborator if we choose to further pursue criminal justice opportunities. In our work on the Open Philanthropy Project (previously known as GiveWell Labs), we use different criteria than for our traditional top charities. In particular, policy-oriented organizations are less likely to engage in work that lends itself to establishing an easily reproducible track record than organizations providing direct aid, so we expect to make riskier bets that may require more heavily weighting intuition and intangibles. At this stage, we are still novices in U.S. policy and politics, so when thinking about making a grant in this area, we heavily prioritize our perceived likelihood of learning from the collaboration with our grantee. More information about the Open Philanthropy Project Published: July 2014 ## Summary What do they do? The Pew Public Safety Performance Project aims "to advance data-driven, fiscally sound policies and practices in the criminal and juvenile justice systems that protect public safety, hold offenders accountable, and control corrections costs" through technical assistance to states, research and public education, and promotion of nontraditional alliances and collaboration around smart criminal justice policies.1 In particular, PSPP seeks to help states pass data-driven policies that reduce incarceration and correctional spending while maintaining or improving public safety and concentrating prison beds on high level offenders. According to PSPP’s mission statement, the project and its partners “identify drivers of cost and growth and conduct system assessments to identify options for reform, drawing on solid research, promising approaches and best practices in states. The project also helps state officials, practitioners, and others across the country share state‐of‐the‐art knowledge and innovative ideas through national research and publications; policy evaluations; public opinion surveys; national, regional, multi‐state, and state‐level policy forums; and online information about what works.”2 What is the track record? Through 2013, PSPP has facilitated or provided technical assistance on twenty-eight legislative reform packages that have been passed in twenty-six states.3 On average, PSPP forecasts that in the states it assists, prison populations will fall about 11% in the five years after reform relative to what would have happened in the absence of reform, resulting in the savings of several billion dollars. We are quite uncertain about the actual counterfactual impact of PSPP's work because in many states where PSPP did not provide intensive assistance, prison growth reversed or stagnated on its own. However, we believe that PSPP increased the probability of reform and/or the quality of reforms in at least several of the states in which it worked. What do we get for our dollar? We have not invested much time in explicit estimates of PSPP's cost effectiveness. We highly value the unquantified benefits of learning from PSPP and we do not believe policy-oriented philanthropy is likely to consist of proven, repeatable interventions with easily quantified expected impact.4 We strongly emphasize the difficulty of quantifying the impact of policy work and advise our readers to treat these calculations as a back of the envelope check on whether PSPP's cost effectiveness is in the same ballpark as other opportunities and not as a literal estimate of PSPP's impact. Based in part on these calculations, however, we do believe that PSPP's past work is plausibly competitive with donations to our top charities, depending largely on how much of the impact of the reforms is (a) accurately estimated relative to the counterfactual and (b) attributable to PSPP. Plans for the next two years and room for more funding. Good Ventures is providing PSPP with$3 million over the next two years, an amount which is less than 25% of the project’s total biennial budget. Over the coming years, PSPP plans to work in additional states, help states to cement existing reforms, explore the potential for reform at the federal level, and continue pursuing research and public education and engaging with nontraditional allies of reform. We do not know which portion of this work would not occur without this funding. We add that collaborating with PSPP seems to us to be a particularly good learning opportunity because:

• PSPP has a track record in multiple states of working intensively on policy changes that occurred within a year of PSPP's arrival.
• We have found PSPP to be particularly easy to communicate with. They have been very open to sharing information with us and we have consistently learned from our conversations with them.
• Policy generalists we have spoken with believe criminal justice reform is an area where there might be continued opportunities for policy reform in the near term, which we expect to lead to more learning opportunities.5

## Our review process

To date, our investigation process has consisted of:

• An initial broad conversation about criminal justice reform with Adam Gelb, PSPP's director, as part of our overall investigation of that cause.
• A second conversation with Mr. Gelb focused on conservative interest in criminal justice reform and on PSPP's work.
• Seeking PSPP's advice on other giving opportunities related to criminal justice reform.
• A meeting with PSPP and other Pew staff in Washington, D.C.
• Reviewing documents PSPP provided including projections of the impacts of the bills they assisted on, data on prison populations before and after the bills were passed, and policy briefs on reforms in several states.
• Vetting PSPP's projections of the impact of one of the bills they helped to pass.
• Speaking to two policymakers who had worked with PSPP in order to improve our understanding of PSPP's role in the legislative process.
• Comparing data in states where PSPP worked to data from states where they did not work.
• Reviewing LaVigne et al 2014, the Urban Institute's assessment of the Justice Reinvestment Initiative, the partnership with the federal government which encompasses much of PSPP's technical assistance to states.
• Meeting with Mr. Gelb and two other Pew officials at our office in San Francisco to discuss PSPP's track record, and determine the details of our collaboration.
• Multiple additional conversations with PSPP about criminal justice reform, their work, transparency, and this review.

## What does PSPP do?

PSPP's goal is to "work with states to advance data-driven, fiscally sound policies and practices in the criminal and juvenile justice systems that protect public safety, hold offenders accountable, and control corrections costs."6 By advancing data-driven policies in their state work and research, PSPP additionally seeks to reshape the framing of the criminal justice debate from "tough on crime v. soft on crime" to a dialogue about finding the best public safety return on investment.7

Activities PSPP engages in to accomplish this include:

1. Research and public education.8 PSPP has published research on:
• The size and cost of federal and state corrections systems
• Lengths of stay in prison9
• "The high cost and low return of incarcerating low-level offenders
• State-by-state recidivism rates
• Effective strategies to reduce prison populations, recidivism, and criminal justice costs in ways consistent with public safety
• Public opinion, both on criminal justice reform broadly and on specific policy proposals"10
2. Engaging nontraditional allies of criminal justice reform.11 PSPP supports (including via financial support) the voices of nontraditional allies of reform such as business leaders, crime victims, law enforcement, and conservative organizations such as the Texas Public Policy Foundation and Prison Fellowship Ministries, and the American Conservative Union Foundation, which together form the Right on Crime initiative.12 PSPP believes that publicizing public support for criminal justice reform (including the support of conservatives) puts reform on policymakers' agendas and expands the window of reforms that policymakers believe are possible.13 In our conversations with U.S. public policy generalists, criminal justice reform specialists, and policymakers involved in reforms assisted by PSPP, Right on Crime came up as a particularly influential organization.14
3. Providing technical assistance to individual states. 15 We discuss this activity in more detail below.

### PSPP's provision of technical assistance to states

Pew provides technical assistance as part of a formal process in partnership with the U.S. government. It states:

"PSPP supports efforts in select states that want a better public safety return on their corrections spending. The project also helps state officials, practitioners and others across the country share state-of-the-art knowledge and innovative ideas through national research and publications; policy evaluations; public opinion surveys; national, regional, multi-state, and state-level policy forums; and online information about what works to reduce recidivism.

Much of Pew’s work in the states is conducted in a partnership with the U.S. Department of Justice’s Bureau of Justice Assistance (BJA) under what is called the Justice Reinvestment Initiative (JRI). Although the first three years of the justice reinvestment process (2007-2009) relied in part on BJA discretionary funding, the program was formalized with an appropriation from Congress in 2010. It is important to note that much of the public material about JRI (including a recent assessment of JRI by the Urban Institute) refers only to the work conducted under the formal federal program; this accounts for discrepancies in various documents about how many states have participated in the initiative and what the total projected savings are.

Between 2007 and 2009, technical assistance to the states was provided by the Council of State Governments Justice Center (CSG) and the Vera Institute of Justice (Vera), with support from Pew and BJA. As a result of increasing demands for assistance from the states, Pew built the staff and organizational infrastructure to provide technical assistance to the states directly, beginning with South Carolina in 2009 for the 2010 legislative session.

Under the formal JRI process, state technical assistance currently is provided in two phases: Phase I, which encompasses analysis of the state’s criminal justice system and development of reform policies up to and including adoption of those reforms (i.e. passage of new laws, court rules and agency policies); and Phase II, which refers to the implementation of the new policies, practices and programs. States are selected for participation in JRI by BJA and Pew, through close consultation and collaboration but independent assessment and decision-making processes. Currently, Phase I technical assistance is provided by Pew and by CSG (with support from Pew and BJA). Phase II assistance is provided by CSG and Vera with funding from BJA."16

While a fairly wide array of reforms fall under the rubric of "justice reinvestment," the approach generally involves using data to improve the efficiency and reduce the costs of the criminal justice system and then using those cost savings to fund evidence based public safety practices and improve state budgets.17 Procedurally, justice reinvestment focuses on "consensus-based, bipartisan, and interbranch decision-making."18

#### What is PSPP's role in the states where it provides technical assistance?

The justice reinvestment process begins with the establishment of an interbranch, bipartisan criminal justice working group composed of policymakers and criminal justice system leaders.19 PSPP's technical assistance then includes three phases

• First, PSPP conducts the data analysis and systems assessment needed for the working group to identify and report on the drivers of correctional populations and costs.20
• Next, using the findings from the first phase, PSPP helps the working group develop policy proposals and is responsible for forecasting the effects of different proposals so that the working group can consider the relative merits of different reforms.21
• Finally, PSPP provides legislative support and public education, aiming to help policymakers move the bill through the legislative process.22

While PSPP's role varies somewhat by state, policymakers in two states where a justice reinvestment bill passed with PSPP's assistance23 told us about specific ways PSPP assisted the reform process:

• hosting a conference to educate policymakers about sentencing and corrections research and the experiences of other states24
• analyzing data to determine what crimes accounted for the majority of the prison population and, in turn, how prison populations would be affected by modifying various criminal justice policies25
• helping the state prioritize among possible reforms and providing information about evidence based practices26
• helping to draft a bill to codify reforms27
• setting up meetings between legislators, the bill's supporters, and criminal justice system stakeholders, such as law enforcement28
• bringing experts to the state to build support for the bill and speak about other states' experiences with reform29
• garnering positive press coverage by engaging with local media30
• estimating the impact of various proposals and amendments on the state's prison populations during legislative negotiations,31 and meeting with legislators and answering their questions about the bill32

#### Which states have gone through the justice reinvestment process?

Between 2007 and 2013, twenty-seven states passed justice reinvestment bills and two states went through the process twice for a total of twenty-nine bills.33 PSPP either funded or itself provided technical assistance to each state other than Delaware, whose technical assistance was provided by the Vera Institute of Justice and funded entirely by the federal Bureau of Justice Assistance.34 The content of the bills varied; PSPP classifies twenty-three of the reforms as "comprehensive" and five as limited.35 In 2014, PSPP is providing assistance to two new states (Mississippi and Idaho) and one state in which a bill previously passed (Michigan).36 In order to receive justice reinvestment assistance, these states had to demonstrate their need for reform (as evidenced by large increases in corrections populations and costs) and that their leadership was committed to the reform process.37 A table listing the state, year, and characteristics of each justice reinvestment bill is available here.

#### What types of policy changes have been made in justice reinvestment states?

The reforms generated by justice reinvestment aim to be individualized and heavily influenced by policymakers' and stakeholders' preferences, and the technical assistance provider's analysis of the drivers of correctional populations and costs in that state.38 PSPP does, however, aim to maintain a focus on evidence-based corrections and sentencing reforms, and does not emphasize other aspects of the criminal justice system. 39 The Urban Institute's recent assessment of justice reinvestment listed the following common reforms:

• Risk and needs assessments of offenders to improve decisions about detention, incarceration, release, supervision, and treatment
• Accountability measures to ensure the use of evidence based practices, require justification of departures from sentencing guidelines, and/or develop data reporting requirements
• Good time credits that reduce the sentences or length of community supervision for inmates who behave well in prison or participate in programs
• Intermediate and graduated sanctions, often based on Hawaii's HOPE model, to deter bad behavior
• Increased funding for community based treatment such as substance abuse programs and reentry plans
• Sentencing changes
• Mandatory supervision requirements to ensure that high-risk offenders are supervised after being released from prison
• Problem solving courts, which are often focused on offenders with substance abuse or mental health disorders
• Streamlined and expanded parole to reduce incarceration while protecting public safety by expanding post-release supervision.40

To get an extremely rough sense of the relative magnitude of these policies, we look to the reform package in Arkansas, the state for which we have the most detail because we chose to more closely vet PSPP's forecasting process there.41 In Arkansas, 74% of the forecasted reduction in prison beds came from three policy reforms:42

• Improving the parole release process to encourage compliance with Arkansas' presumptive parole system (a system where, absent a disciplinary record or failure to satisfy special conditions, offenders are presumed to be transferred from prison to parole after a certain proportion of their sentence is served) and ensure that parole hearings occur on time43
• Instituting earned discharge for parole so that parolees are incentivized to follow the rules and rewarded for doing so with accelerated discharge from supervision44
• Measures to encourage judges to adhere to sentencing guidelines45

### Future activities

Pew may broaden its scope in the future, as discussed below

## Track record

Our vet of PSPP's track record focuses on evaluating the impact of bills associated with its technical assistance activities discussed above. This does not mean we think PSPP's other activities aren’t valuable. Rather, consistent with the Open Philanthropy Project's emphasis on "giving as investment", our investigation of PSPP's track record was geared towards determining whether PSPP has a generally impressive record of playing an important role in meaningful policy wins, rather than towards determining the precise "amount" of policy gains we can expect per dollar. To the extent that PSPP has such a track record, we believe a grant to PSPP as a whole (including its harder-to-evaluate activities) is likely to be a good investment. We focus on two main (not entirely separable) questions:

1. How meaningful were the bills that passed in the states where PSPP worked?
2. How much of a role did PSPP play in the states where it worked? Would large, legislative reform packages have passed in PSPP's absence? If so, to what extent did PSPP improve the quality of the bills?

We recognize that one obstacle to evaluating PSPP's track record is that it may be too soon to evaluate the success of the most recent reforms. This issue is especially important because PSPP's assistance has evolved over time and these recent reforms may better reflect PSPP's future plans than work that occurred farther in the past.46

### How do we evaluate the success of criminal justice reforms?

When evaluating reforms to the criminal justice system, we are primarily concerned with three outcomes.

1. First, we value reductions in the number of people incarcerated because of the large human and economic costs of incarceration. Although we primarily discuss the most easily available data, which focuses on overall prison populations, we are especially interested in reducing the number of low-risk and non-violent offenders in prison. (PSPP is currently developing performance measures that account for the composition of the prison population as well as its magnitude.).47
2. Second, we are concerned with criminal justice reform's impact on public safety and in particular on violent crime. We did not closely vet these bills' effects on crime rates, but justice reinvestment aims to reduce recidivism by cutting prison costs and redirecting savings to improve reentry and community supervision.48 PSPP is currently developing a risk- and severity-weighted measure of recidivism and we expect this measure will be a decent proxy for public safety.49
3. Third, we value criminal justice reform's positive impact on state budgets. According to a Pew report, total state spending on corrections increased from $12 billion in 1987 to$49 billion in 2007.50A more recent report by Pew estimated that, on average, the operating cost to incarcerate a prisoner for one year was almost $29,000 in 2008.51 Our examination of PSPP's track record focuses on the first of these factors -- the organization's impact on prison populations. This is the legislation's most direct effect; if the bills PSPP assisted have a substantial impact, this is where we would expect it to show up most quickly. The bills' effects on prison population are also the easiest to measure and the outcome for which we have the best data. Moreover, because decreases in prison population directly cause decreases in correctional spending, we would expect the bills' effects on these outcomes to be very highly correlated. A portion of this decrease in correctional spending is reinvested in programs to increase public safety and the rest is used to improve state budgets. We have not closely looked into the amount of savings that are typically reinvested nor have we closely vetted the evidence behind these programs. A full review at the level of depth we investigate our top charities would have looked into these issues and would also have more closely looked into changes in crime rates, and in the composition of the prison population. As noted above, we would guess that there will not be a negative effect on either of these outcomes due to PSPP's attention to improving community supervision and limiting prison beds for high level offenders. ### Were the bills PSPP worked on a big improvement over the status quo? #### How big of an impact does PSPP forecast for the bills it assisted? States seeking to engage PSPP are assessed in advance for their need for reform. This assessment has a particular focus on whether prison populations and correctional costs are rising, so a naïve comparison between pre-reform and post-reform prison populations would be unlikely to yield an unbiased estimate of reform's effects.52 Instead, to get an initial sense of the bills' magnitude, we look at PSPP's impact forecasts (while cautiously interpreting the organization's forecasts of its own impact).53 For each state PSPP assists, PSPP (or a contractor) creates or verifies a state's own a) projections of the amount that prison populations would grow in the absence of reform and b) projections of post-reform populations. PSPP shared with us results of these forecasts from eighteen of the twenty-nine reforms that occurred before 2013.54 Prison populations were rising in these states prior to reform. On average, prison populations grew by about 10% over the course of the five years leading up to reform's passage and PSPP projected further growth of 11% over 5 years if the status quo were to be maintained.55 PSPP forecast that, with reforms, prison populations would, instead, decrease by an average of 1.7% over the five years after reform took place. This would lead to prison populations that would on average be 11% lower after reform than they would have been if the status quo were maintained. We believe that these forecasted prison population reductions due to bills that PSPP assisted are PSPP's most significant claimed successes, so our evaluation of PSPP's impact focuses on vetting these forecasts.56 We did not ask PSPP for estimates of the fiscal savings in each state because we would guess that correctional cost savings are roughly proportional to the number of prison beds eliminated. That said, PSPP did send us policy briefs or other research materials with estimates of fiscal savings from five states: • Arkansas:$875 million over ten years57
• Georgia: at least $264 million over five years58 • Hawaii:$130 million over five years59
• Kentucky: $422 million over ten years60 • South Carolina:$241 million over five years61

We have not carefully vetted these estimates.

#### Vetting the forecasts

The majority of saved prison beds attributed by PSPP to these bills represent averted potential prison population growth and not absolute declines in a state's prison population. This complicates our attempts to verify the accuracy of the impact forecasts. Prison populations could drop by the predicted amount in the states where PSPP works, but these small absolute declines only reflect major changes (relative to the counterfactual) if the counterfactual forecasts of continued growth under the status quo are correct. In light of these concerns, we took the following steps to investigate the impact on prison populations of the bills PSPP worked on:

• For a fairly arbitrarily chosen state (Arkansas), we looked into the methods and assumptions behind PSPP's forecasts and attempted to determine whether they were reasonable, keeping in mind that we are not experts on criminal justice reform.62
• We plotted PSPP's forecasts of post-reform prison population next to actual post-reform prison population in these states. We looked to see whether actual post-reform prison populations were reasonably close to PSPP's forecasts and to see whether there were noticeable breaks in prison population trends at the time various bills passed.
• We used data from the Bureau of Justice Statistics to graph actual prison population data in each state in which PSPP worked and examined these graphs to look for breaks in the trends at the time of reform.
• We calculated actual post-reform prison population changes in each state, including both states PSPP had worked in and states PSPP had not worked in.

#### Vetting the model used to forecast the impact of reform in Arkansas

PSPP contracted out the forecast of the impact of SB 750, Arkansas' justice reinvestment bill.63 The impact estimate is composed of two separate forecasts. First, there is a "cost of doing nothing" forecast that projects future prison populations under the status quo. Second, PSPP estimated the reduction in inmates' time served due to the bill.64

Between 2008 and 2010, Arkansas' prison population grew 9.9% from 14,686 to 16,147.65 The forecasts projected that, had no bill passed, Arkansas' prison population would have continued rising to 19,222 in 2015.66 Instead, due to SB 750's passage, PSPP predicted that Arkansas' prison population would fall to 14,226 in 2011, followed by a slower rise to 16,279 in 2015.67Thus, PSPP forecast that five years after its passage, the bill would be saving Arkansas 2,943 prison beds – a 15.3% decline relative to the counterfactual.68 PSPP reports that they intend to be conservative as they settle on assumptions of the effects of each policy change.69 Their level of conservatism depends on three factors:

• How reliable is the data related to that policy change?
• How "hard" is the policy lever used to create change (i.e. how likely are practitioners to implement the reform as intended)?
• How much confidence does PSPP have in the likelihood of quality implementation?70

A couple of notes about the overall model behind the Arkansas forecast:

• Prison beds saved are allocated to the year the inmate with reduced length of stay is admitted into prison, not the year in which the bed is actually saved. This led the model to overstate the speed with which the Arkansas reforms would take effect.71
• The model does not explicitly account for most of the second order effects of reform. For example, the model does not explicitly model recidivism among the inmates whose sentences are reduced, any change in the crime rate or the composition of crime due to changed incentives, or behavior change among law enforcement in response to reduced lengths of stay. We are unsure whether these omissions are more likely to lead to underestimates or overestimates of the bill's impact on prison populations.72

Our bottom line is that there are clear limitations to the general model PSPP used to forecast the effects of Arkansas' reform and we would expect to be more confident in PSPP's more recent microsimulations (which address the above issues) than we are in this older model. Overall, we would guess that the Arkansas model's limitations would lead PSPP to overestimate the speed with which the bill affected prison populations by up to a couple of years. However, assuming here for the sake of argument that the assumptions of the impact of individual provisions of the policy are correct, we would guess that the model's limitations would not lead to a large overestimate of overall impact.

#### Vetting PSPP's estimates of the impact of individual policy changes.

PSPP estimated that Arkansas' justice reinvestment bill saved 2,541 prison beds in the bill's first year.73 Because PSPP projected that 74% of these saved beds resulted from three policy changes (see above), we focused on understanding how PSPP and Arkansas officials arrived at their projections of these three reforms.

• Improving Arkansas' parole release process. While analyzing the drivers of increased correctional costs in Arkansas, the state's criminal justice working group, assisted by PSPP, found that 55% of prisoners were being held an average of 6.2 months beyond the date at which they were presumptively entitled to parole meaning that absent a disciplinary record or failure to satisfy special conditions, there was a presumption that they would be released to parole on that date.74 While the state did not record a reason for most of these delays, PSPP believes some of these prisoners were held because (due to crowding) they were being kept not in prisons but in jails (where parole review is often delayed), some were held because of the parole board's failure to properly implement presumptive parole, and some were held because the inmates had behavioral violations in prison or otherwise did not deserve parole.75 SB 750 included provisions to address this issue by:
• Reemphasizing the expectation that, absent behavioral violations or other circumstances, Arkansas maintains a presumptive parole system.
• Establishing new reporting requirements for the parole board including the responsibility to track and report on the percent of offenders receiving their parole hearing on time and the percent of offenders approved for parole at their hearings
• Establishing a system allowing sentenced offenders held in jail to be transferred to parole even if they never entered a prison operated by the state Department of Corrections.
• Instituting additional training, oversight, and guidelines for parole officers
• Initiating the process of preparing for an offender's hearing further ahead of that hearing's date76

PSPP estimated that this set of policies would cut both the percent of prisoners held beyond their presumptive parole date and the length for which those prisoners are held in half – to 25% and three months respectively – equivalent to a 78% reduction in the number of prison beds devoted to prisoners held past their parole dates.77 Overall, PSPP estimated that these policy changes would save 967 prison beds in 2011, which is 5.8% of the baseline forecasted prison population.78 PSPP told us that by six months after the bill was passed, the state had progressed halfway to PSPP's predictions, with 47% of inmates held for an average of 4.2 months beyond their parole date.79 We do not have the expertise to determine whether PSPP's assumption that Arkansas' attempt to streamline the parole process will reduce the total amount of time prisoners are held past their parole hearing dates by 78%. The impact of the relevant policy changes is far from mechanical and could easily be reversed by a Governor unfriendly to reform. We would therefore guess that PSPP's assumption is somewhat optimistic but based on the progress made after six months, it seems to be a good faith attempt and unlikely to be a major overestimate.

• Earned discharge from parole. This policy allows Arkansas' Department of Community Corrections to count each day of parole a parolee serves without misbehavior as two days towards the parole that they owe. In effect, the policy halves parole time for well-behaved parolees.80 If an offender commits a new felony while on parole, his/her parole can be revoked and s/he can serve his/her remaining parole time in prison as well as being prosecuted for the new crime as such.81 Instituting earned discharge for parole aims to save prison beds by removing parolees from supervision more quickly so that they are less likely to have this additional time added to their sentences if they commit another felony.82 Using the total number of parolees who had their parole revoked for committing a new crime during the second half of their parole in 2009 as a baseline, PSPP estimated that, in 2011, this policy change would save 396 beds.83 396 beds saved represents 2.4% of the baseline forecasted prison population in 2011.84 We note that the prison beds saved due to this policy come from reducing the sentences of repeat offenders who may be more likely than others to commit future offenses. However, the calculation of this policy's impact strikes us as fairly mechanical and it is hard for us to imagine it as being a large overestimate.85
• Encouraging adherence to sentencing guidelines. PSPP forecast that Arkansas would save 674 beds in 2011 (4% of the forecasted prison population under baseline conditions) by encouraging judges to adhere to Arkansas' sentencing guidelines.86 The bill encourages judges to sentence within the guidelines by a) merging the form where the sentence is recorded with a second form the judge must use to justify any deviation from the guidelines – so that any deviation from the guidelines is salient to the judge at the time the sentencing decision is made, and b) mandating the public reporting of which counties deviate from sentencing guidelines most frequently.87 PSPP assumed that this policy would have the effect of reducing the sentences for lower-level crimes to their total level in 2002 (a 20% reduction).88 While we are highly uncertain about the extent to which judges are affected by this type of behavioral nudge, PSPP's estimate, which is just 16.8% of the beds that would be saved by complete adherence to guidelines, strikes us as reasonable.89

Overall, we do not have the subject matter expertise to come to a confident conclusion about whether PSPP's assumptions about the effects of each policy change in Arkansas' legislative package are reasonable. However, while flaws in PSPP's model may exist, PSPP's approach to forecasting the effects of Arkansas' reform strikes us as reasonable and it appears to us that PSPP had thought of and addressed most of the concerns we could come up with. We believe there is a high risk of relying too heavily on an organization's own forecasts of its impact, but our vet of PSPP's forecasting process left us reasonably impressed. We thus would guess that PSPP's forecasts are reasonable representations of the projected impacts of the reforms passed.

#### Comparing PSPP's forecasts to actual post-reform prison population data

Looking in depth at post-reform events in Arkansas Informal evidence suggests that SB 750 had a substantial initial impact on Arkansas' prison population. The bill was signed into law in March, 2011.90 Between 2011 and 2012, Arkansas' prison population fell by 8.9%, which is a slightly greater decline than predicted by PSPP.91 However, this downtrend was reversed as the prison population rapidly increased by about 1,400 inmates in the three months from June to September 2013.92 While we have not investigated this issue in depth, journalistic accounts and conversations with PSPP both suggested that the increased prison population was caused by policy changes made by the Board of Corrections in response to a murder allegedly committed by a parolee who was arrested and charged with felonies repeatedly without having his parole revoked.93 These events somewhat reduce our confidence that gains achieved by PSPP will be sustained in the long run (although it is also possible that increased incarceration would have been even greater absent the reforms PSPP worked on). Some of the reforms PSPP assists rely heavily on buy-in from implementers and may face a heavy risk of reversal when a high profile crime occurs.

Post-reform data from additional states In this section, we compare the forecasts of post-reform prison populations to actual post-reform data to further hone our sense of the general reasonableness of PSPP's forecasts. It is important to emphasize that actual data in line with the forecasts would only imply that PSPP's impact estimates are accurate if the organization's baseline forecasts of continued growth in the absence of reform were accurate, something we are not sure of, as discussed in a later section. We also interpret this data cautiously because some states' reforms were quite recent, which means that their full effects may not yet be visible but also means it is too early to tell if those reforms will be sustained. PSPP provided us with both a) forecasts of post-reform prison populations and b) actual post-reform data for fifteen reforms. In addition, PSPP pointed us towards the Urban Institute's publication of data from South Carolina.94

Graphs of actual prison populations and projected prison populations (with and without reform)
Vertical lines represent year of reform.

South Dakota

Oregon

Kansas (2013)

West Virginia

Hawaii

Georgia

Pennsylvania

Missouri

Kentucky

Ohio

North Carolina

Arkansas

New Hampshire

Vermont

Arizona

Rhode Island

Kansas (2007)

Texas

South Carolina

For these fifteen reforms, the projections seem to have tracked well with later outcomes, albeit somewhat more likely to underestimate than overestimate the post-reform prison population.95

#### Comparing pre-reform trends to post-reform data

Graphs of pre- and post-reform prison population data could reveal breaks in trend that would be evidence of impact. Again, vertical lines represent year of reform.

96 We view this as a very back of the envelope test for impact because we do not have strong priors about the dynamics of reform (i.e. how quickly a successful bill would lead to a decrease in prison populations), nor do we know the exact effective date at which policy changes are phased in. We only have annual data for most states, and often have few years of post-reform data. With those caveats in mind, based on eyeballing these graphs, there appears to be a reasonable case that reform was followed by a meaningful change in trend in five or six states (Hawaii, Connecticut, North Carolina, Rhode Island, South Carolina, and perhaps Nevada).97 In four states (Georgia, Delaware, Oklahoma, and Missouri) it appears to be too soon to tell whether there was a break in trend post-reform. In the remaining fourteen states, we do not see a noticeable change after reform took place. The below table summarizes this data:

Taking a mean across all states with justice reinvestment bills, prison populations declined, on average, by 1.9% between December 31 of the year the bill passed and December 31, 2012. This result is difficult to interpret because it combines data from states less than a year out from reform with data from states reformed five years ago. Therefore, for each justice reinvestment state, we also report changes in prison population between December 31 of the year before reform, and December 31 of each following year. Taking a mean across states, we see that prison populations fell between 0.7% and 1.8% between the year before reform and 1-4 years afterwards.98 Overall, the average size of absolute declines in prison populations following the passage of justice reinvestment bills are consistent with the approximately 1.7% average absolute decline forecast by PSPP. Our beliefs about the overall impact of these bills therefore depend heavily on the counterfactual of what potential growth was actually averted and what would have happened in the bills' absence.

#### Comparing post-reform prison growth to data from states where PSPP did not provide assistance

In this section, we examine states PSPP did not assist to provide some context on whether prison population growth in those states PSPP did assist might have stagnated or reversed even if PSPP had not provided assistance. Five factors cause us to exercise particular caution during this analysis:

1. States that receive assistance from PSPP are selected, in part, based on their high need for reform (in terms of rising corrections populations and costs).99 Therefore, even if states that did not receive assistance saw correctional populations decline, states that did receive assistance may have been particularly likely to experience continued growth in the absence of reform. We are somewhat skeptical of this factor's importance, however, because prison populations seem to have been rising almost as quickly in states that did not receive PSPP's assistance as in those states that did.100
2. States that receive assistance from PSPP may differ from other states because they are required to demonstrate their commitment to the justice reinvestment process.101 We would guess that this cuts in the opposite direction from the previous factor and suggests that the states PSPP worked in were more likely to pursue reform in PSPP's absence than the typical state. On the other hand, PSPP often develops relationships with state leaders, aiming to raise awareness of research and other state reforms, and build the political support necessary to adopt comprehensive reforms.
3. States' justice systems, criminal penalty laws, and political environments vary greatly and many of the factors that best explain variations in state prison population growth may be idiosyncratic shocks, so population declines in some states may not be highly predictive of population declines in others. For example, PSPP stated to us that California experienced a large, idiosyncratic decline in its prison population due to Brown v. Plata, a Supreme Court case ordering the state to address prison overcrowding, and New York experienced a large idiosyncratic decline due to an NYPD policy of pursuing fewer felony arrests in New York City. PSPP also has observed to us that counts of sentenced prisoners still were rising in 24 states as of 2012, and stated that with so much variety and so many idiosyncrasies, comparisons of trends can raise questions but not ultimately provide answers as to why prison populations have followed the patterns they have.102
4. PSPP's technical assistance in the states where it worked intensively –– and its other work, including organizing convenings, publishing research papers, general media outreach, and promoting nontraditional allies of reform –– may have affected the national dialogue on criminal justice issues and been responsible for some of the reduced prison growth in other states.103 If PSPP's efforts in states where it provided intensive assistance had positive spillover effects on other states by, for example, setting a positive example of reform, our comparison between states where PSPP provided intensive assistance and other states would yield a misleading picture of the impact of PSPP's assistance. Specifically, spillover effects that were in fact magnifying PSPP's impact could be misleadingly framed as evidence against PSPP's impact. PSPP states: "There may be some national trends in policy making by governors and legislators, and in case-level decision making by individual judges, prosecutors, parole board members, probation officers and other system actors, that are influenced by an overall climate of reform and general attitudes toward crime and punishment. While Pew, JRI partners and others working in the field cannot claim anything near exclusive credit for creating an environment for reform, changing the national conversation from “tough vs. soft on crime” to “public safety return on investment” is a key goal of our broader work. This work has included hundreds of research publications, extensive media outreach, and scores of convenings with leading policy makers across the country. These national efforts “contaminate” the group of non-JRI states by exposing policy makers in those states to justice reinvestment data and strategies. Comparing them to the intensive assistance states effectively counts these broader national efforts against the impact of work in intensive JRI states. This is unfortunate since any policy change effort should seek to maximize change with available resources."104
5. PSPP provided limited assistance in some states that did not pass justice reinvestment bills, including California, Florida and Colorado, which may have contributed to declines in the prison population in those comparison states. PSPP stated to us that "Beyond these broad national impacts, Pew and its partners have provided limited assistance in some states, including California, Florida and Colorado. Counting these states among the non-JRI group further complicates comparison of JRI vs. non-JRI states. California, for instance, picked up the Performance Incentive Funding (PIF) concept from 2008 JRI work in Arizona and fashioned it with very light assistance from PSPP into SB 678 of 2009. In its first year of implementation, the California PIF program cut the number of probation revocations to state prison by nearly 6,200, nearly the identical number by which the state’s prison population dropped from 2009 to 2010."105

Keeping the above caveats in mind, below we compare the aggregate prison population in states PSPP intensively assisted to the aggregate prison population among all other states.

108

The reversal of rapid prison population growth in states unassisted by PSPP initially suggests to us that growth may have subsided absent PSPP's intensive assistance. When growth in all states is weighted equally (to reduce the potential dominance of idiosyncratic factors in California and New York), the decline in prison population is less pronounced, but the pattern of population growth subsiding in 2007-2008 remains.109

As an additional way to compare post-reform changes in prison population to changes in other states, for each state, we report below a) the percent change in prison population each state experienced between December 31 of the year before that state was reformed and December 31, 2012; b) the change in prison population among states that had not received assistance from PSPP by the end of 2012 over the same time period; and c) the difference between (a) and (b). We do this exercise including and also excluding New York and California from our measure of states unassisted by PSPP. Thus, we look at whether, after reform, states' prison populations decreased relative to the rest of the country.

Comparison between prison growth in JRI states and non-JRI states over similar time frames

On average, prison populations grow post-reform relative to states that did not receive assistance. When New York and California are excluded, prison populations decline by a mean of 1.1 percentage points relative to populations in the remaining states that did not receive assistance.

What might explain the decline in prison population growth in states that did not receive PSPP's assistance?

We take the reversal of prison growth in states that did not go through the justice reinvestment process to suggest the possibility that in the states PSPP assisted a) reform might have occurred in PSPP's absence or b) prison population growth might have reversed without explicit policy change. We have not investigated the possibility that there are causes of declining populations in non-justice reinvestment states that would have left behind the states PSPP assisted. We did, however, ask Mr. Gelb for alternative explanations for changing trends in non-justice-reinvestment states. He stated that: 112

• As discussed above, idiosyncratic factors such as the Plata decision in California and NYPD policy in New York, systematic differences between states PSPP assisted and other states (i.e. selection effects), positive spillover effects from PSPP's intensive assistance, and PSPP's national research reports, conferences, partnerships, and other work all could have made contributions.
• Colorado passed a series of legislative reforms similar to the ones PSPP has assisted with but no other state passed a comprehensive criminal justice reform package without assistance from PSPP or the larger Justice Reinvestment Initiative co-funded by the Justice Department.113
• Changes in non-justice reinvestment states might be attributable to executive, agency level, judicial, or prosecutorial policy changes. These changes may be more easily reversible than the legislative changes achieved by PSPP.
• Depending on how prison populations are counted, increasing jail populations may substitute for declining prison populations, so incarceration overall may not have decreased.
• PSPP's technical assistance may have had positive spillover effects or PSPP's research, public education, and support for nontraditional allies of reform may have led to population decreases in states where PSPP did not provide intensive assistance.
• A temporary response to the Great Recession. We would guess that this last possibility would not apply differentially to states that did not engage PSPP, so, were it to be true, it would probably not affect our overall conclusion.

Our bottom line on comparing states that engaged PSPP with states that did not do so

Comparing states that received intensive assistance from PSPP to states that did not lowered our confidence in the likely magnitude of the bills PSPP worked on. This comparison particularly called into question the idea that the "cost of doing nothing" would be as high as projected by PSPP and raised the possibility that potentially rising prison populations may have been averted by underlying dynamics (such as state fiscal problems, declining crime rates, increasing conservative support for reform, and policy diffusion among states) independent of PSPP. Overall, while this analysis raised questions, the data is compatible with the possibility that the bills PSPP worked on had a large impact on prison populations as well as the possibility that they averted little, if any, growth. If we pursue criminal justice reform in considerable depth, we plan to look into this issue further.

### What was PSPP's role in the bills it assisted?

In addition to vetting the effects of the bills PSPP assisted, we also investigated PSPP's effect on those bills. In particular, we sought to learn a) whether PSPP was responsible for the passage of legislation that would otherwise not have occurred and b) whether PSPP improved the quality of legislation that otherwise would have passed in some form. To efficiently investigate these questions, we asked PSPP to refer us to policymakers who had worked closely with PSPP on justice reinvestment bills and could speak to that experience. They referred us to Grant Tennille, the Deputy Chief of Staff to Arkansas Governor Mike Beebe during the time of Arkansas' reform, and Jay Neal, vice-chair of the Georgia State House of Representatives' Public Safety Committee and co-sponsor of Georgia's reform.114 We recognize that PSPP would be unlikely to refer us to policymakers who felt the organization did not play a major role, so our decision to seek referrals in this way means we must interpret these interviews cautiously. Both interviewees did appear to have been key players in their respective bill's passage and were valuable referrals. Mr. Tennille disclosed that PSPP retained Cranford Johnson Robinson Woods (CJRW) to support the state’s criminal justice reform effort through government and public relations work and that Mr. Tennille's wife served as CJRW’s lead staff person.115

#### PSPP's role in Georgia according to Representative Neal

This section cites the views of Representative Neal according to our conversation notes.

Georgia's Governor Deal was a strong proponent of criminal justice reform prior to engaging PSPP.116 On the other hand, a conference sponsored by PSPP was directly responsible for Representative Neal's interest in the issue.117 Representative Neal went on to propose Georgia's engagement of PSPP and to sponsor the reform bill.118 PSPP's assistance and data analysis helped the state's Special Council on Criminal Justice Reform identify drivers of the prison population, the crimes prisoners had been convicted of, and the lengths of prisoners' sentences.119 For example, with PSPP's assistance, the Council found that "the majority of people in prison in Georgia had committed drug or property crimes and were not violent or criminally minded."120 This type of data helped to reverse Georgians' attitude toward criminal justice.121 PSPP also helped to design and draft Georgia's reform bill and conducted stakeholder outreach and education, for example by bringing organizations such as Right on Crime into the state.122 In addition, PSPP helped to address law enforcement's concerns about the bill.123 It is not clear what would have happened to Georgia's criminal justice system without PSPP's involvement.124 Georgia's Governor, Speaker, and Chief Justice probably would have championed reform in PSPP's absence, but prior to Representative Neal's attendance at PSPP's conference, there was no champion for reform in the General Assembly.125 Representative Neal had not heard anyone in the legislature mention the need for reform prior to PSPP's involvement.126 Moreover, when the state later tried to reform its reentry system without PSPP's assistance, the Criminal Justice Reform Council struggled to design a proposal and had to engage a consultant referred by PSPP.127 Overall, some sort of criminal justice reform would likely have passed in Georgia without PSPP's involvement, but it would have relied less on data and experience from other states and may have been less strong.128

#### PSPP's role in Arkansas according to Grant Tennille

This section cites the views of Mr. Tennille according to our conversation notes.

Mr. Tennille first became involved in Arkansas' criminal justice reform process after the bill was introduced, so he is most familiar with PSPP's role from that point on.129 Governor Beebe believed criminal justice reform was necessary in Arkansas before interacting directly with PSPP.130 PSPP helped Arkansas' working group prioritize among potential reforms by analyzing data to help the state understand what crimes most Arkansas prisoners had been convicted of and how different proposals would affect Arkansas' prison population.131 Once an initial set of reforms was chosen, PSPP wrote an outline of a bill to implement those reforms.132 Over the course of legislative negotiations, PSPP helped to inform stakeholders about the bill, answered technical questions and provided advice about alternative proposals, and spent time in Arkansas meeting with legislators.133 Prior to PSPP's involvement, the main advocates for criminal justice reform in Arkansas were the state's Black Caucus and allies in the Democratic Party, who did not have a concrete proposal on the table and were unlikely to pass anything stronger than the eventual legislation.134 If PSPP had not been involved, a criminal justice reform bill probably would have been proposed but it would not have been based on data or an analysis of Arkansas' criminal code; instead policy changes would have been made "fairly haphazardly without regard for how it would actually affect prison populations."135

#### Our bottom line on our conversations with policymakers and on PSPP's role

While we are cautious not to place too much weight on the views of policymakers who worked closely with PSPP and were referred to us by the organization, we came away from these conversations believing that PSPP played a major role in these reforms.136 Both policymakers we spoke with strongly believed that their states would have struggled to pass high quality legislation without PSPP's assistance. We do note, however, that the governor of each state was interested in reform prior to PSPP's involvement. PSPP's engagement may have been more important for improving the bills' quality than for getting them on the agenda.

### Bottom line on PSPP's track record

Overall, the data are consistent with PSPP's claim that by averting growth and causing slight declines in prison populations, the bills they worked on reduced prison populations (relative to the counterfactual) by an average of 11%. However, the data are also consistent with the possibility that prison growth would have leveled off even without these reforms and that the bills themselves saved a much smaller number of beds. After vetting PSPP's overall forecasting process and many of the assumptions behind the forecasts in Arkansas, we were persuaded that PSPP makes a good faith and reasonable, if perhaps slightly optimistic, attempt to accurately forecast the effects of the policy changes made by these bills. Some of the important policy changes in Arkansas had fairly mechanical effects on the prison population. For example, much would have to go wrong for shortening the length of parole to fail to result in fewer revocations. Where the forecasts required more judgment, we felt that PSPP's judgments were reasonable and had tried to incorporate most issues we, as non-experts, could think of. We also heavily weighed our interviews with two policymakers, both of whom strongly believed that an equally strong and data-driven bill could not have passed without PSPP's assistance.

## Cost effectiveness

Estimating the cost effectiveness of PSPP’s work in the states is challenging due to uncertainties in calculating both costs and benefits. We have explored PSPP's track record (and discussed the difficulties of precisely estimating its impact) above. On the cost side of the equation, PSPP’s budget includes funding for national research, conferences and many other activities beyond direct assistance to states that likely contribute to the impact of PSPP's state assistance. In addition, many of the JRI states received assistance from other providers, which receive funding from the federal government.

Below, we attempt an extremely rough calculation of the cost effectiveness of PSPP's assistance to states as a back of the envelope check on whether funding PSPP would be reasonably cost effective if the bills PSPP assisted with had the effects PSPP claims. As above, our analysis attempts to focus on the benefits and costs of PSPP's technical assistance to states and we do not attempt to quantify the cost effectiveness of its other work. We acknowledge that our ability to separately estimate the cost-effectiveness of PSPP's state assistance is limited, because PSPP's spending on other activities likely contributes to the success of this program.

PSPP's budget is not public but Pew and the Bureau of Justice Assistance together have spent no more than $25 million on state assistance across all states that have participated in the policy development phase of the justice reinvestment process going back to Texas and Kansas in 2006.137 To be conservative, we use this$25 million figure as though it were our estimate of combined Pew/federal spending on state assistance.

PSPP assisted with 29 reforms that passed by the end of 2013 in 27 states. At its peak in 2010, the aggregate prison population among these states was 780,000. Assuming that the impact forecasts accurately predicted each bill's impact on prison populations, we would expect the bills to lead to about an 11% decrease (the average forecasted reduction among states for which we received forecasts from PSPP was 11%). An 11% reduction in the prison population of these 27 states would lead to about 85,800 fewer prisoners at a given point in time.138 If we assume that the gains from reforms last about ten years and we assume that these gains would not have occurred in PSPP's absence, then PSPP's state assistance would have led to about 858,000 fewer person-years in prison. 139

If (as noted above) we assumed for the sake of the calculation that total spending to achieve these gains was $25 million, we would then calculate that Pew and BJA spent about$29 per "prison year averted," again assuming none of the reforms would have happened without PSPP's involvement. In addition, these averted prison years would have led to billions of dollars in savings to state budgets.

We strongly emphasize the difficulty of quantifying the impact of policy work and advise our readers to treat these calculations as a back of the envelope check on whether PSPP's cost effectiveness is in the same ballpark as other opportunities and not as a literal estimate of PSPP's impact. Based in part on these calculations, however, we do believe that PSPP's past work is plausibly competitive with donations to our top charities, depending largely on how much of the impact of the reforms is (a) accurately estimated relative to the counterfactual and (b) attributable to PSPP.

## PSPP's plans for the next two years

PSPP has informed us that its primary objectives for the next two years include:

• Providing intensive technical assistance to additional states seeking to improve their adult or juvenile justice systems. In 2014, these states include Mississippi, Idaho, and Michigan.
• Cementing gains in states that have undergone reform. PSPP will do this by helping states establish oversight councils to hold agencies accountable for implementation and by helping states construct performance measures to track progress. All new states will receive this assistance as well as certain states PSPP has already worked in.
• The oversight councils serve two key functions: 1) to receive the performance measurements and implementation updates from the agencies/entities most responsible for carrying out the new sentencing/corrections policies; vet that information; and hold the agencies accountable, and 2) to educate other public officials, the media and public regarding the progress on implementation and performance; suggest modifications to the policies; and advocate on behalf of resources needed to sustain improvements. The councils can be established formally through legislation or executive order, or less formally through Memoranda of Understanding (MOUs). PSPP intends to help states establish these councils and provide support in their initial meetings. Since many of these states are part of the Justice Reinvestment Initiative, PSPP will also facilitate the work of the “Phase II” (or implementation) assistance.
• Documenting the impact of reforms generated by PSPP. PSPP seeks to complete evaluations of at least four more policies the project has recommended in order to expand the evidentiary base for effective criminal justice policies.
• Producing additional research and publications. Studies planned over the coming years include a national report on the costs and consequences of allowing inmates to serve their full sentences and be released without community supervision and another report presenting multi-state juvenile recidivism rates and discussing best practices.
• Engaging and educating influential constituencies and nontraditional allies of reform. This includes supporting the expansion of Right on Crime and working with the National Sheriffs Association to highlight how the reform process includes sheriffs and local government representatives, and how local jails have benefitted from reform.
• Exploring potential reforms at the federal level. PSPP plans to explore whether there is an opening to make progress in the federal criminal justice system and also plans to advocate for an increase in appropriations to the Justice Reinvestment Initiative.
• Monitoring and evaluation. PSPP's partners plan to ramp up policy evaluations of the Project's work.140

We do not know which, if any, of these activities would be unlikely to happen without GiveWell's support.

## The organization

We find it quite difficult to holistically asses an organization and the people that run it and we have limited observations to assess PSPP. PSPP's Director, Adam Gelb, has communicated extremely well and directly with us, has had thoughtful answers to most of our questions, and has been quite open about sharing substantive information about PSPP. We first spoke with Mr. Gelb as part of our investigation of the cause of criminal justice reform and found that to be one of the most helpful conversations during that investigation. Both policymakers who worked on legislation with PSPP had extremely strong impressions of the organization, although we would (to some degree) expect referrals from PSPP to speak highly of them. PSPP is the organization we have seen whose approach to criminal justice issues – focusing prison beds on high level offenders, aiming to minimize the fiscal and humanitarian costs of the corrections system while maintaining or improving public safety, taking advantage of the unique opportunity to unite liberals and conservatives - matches the vision of the issues that we have identified as most promising. As we enter new fields in which we have little expertise, we believe it is valuable to collaborate with organizations that share our approach and seem strategic. PSPP's self-evaluation is quite strong relative to that of other organizations we have seen, but leaves some questions unanswered, as discussed below. We hope to see PSPP's self-evaluation improve over time and are eager to monitor the results of its development of new performance measures and its focus on the establishment of oversight councils in the states in which it works.

## Open questions

Major unresolved issues include:

• The general difficulty of assessing the impact of an organization whose role is technical assistance and facilitating policy change rather than direct aid. As we previously wrote in our review of Deworm the World, "there are substantial potential advantages to supporting such an organization, as it may be able to have more impact per dollar by influencing government policy than by simply carrying out programs on its own, but this situation also complicates impact assessment."
• The appropriate counterfactual to PSPP's assistance given the decline in aggregate prison growth even among states where PSPP did not work. This is partially separable into four questions:
1. Would a bill have been introduced without PSPP's assistance?
2. If so, would that bill have passed without PSPP's assistance?
3. If a reform bill would have passed without PSPP's assistance, would the bill have been of lower quality?
4. If a similar bill would not have passed without PSPP's help, would some other policy change or non-policy change (such as a decrease in crime) have led to a decline in prison growth?
• The risk that PSPP's work in future states will be less impactful than the work with which it established its track record because PSPP initially prioritized states in which policymakers were particularly committed to the reform process or in which forecasted prison growth was particularly high.141
• The risk that policy gains achieved by PSPP won't be sustained. In one state, Arkansas, we did see evidence that policy gains from a bill that PSPP worked on were reversed after a parolee whose parole was not revoked despite multiple felony arrests allegedly killed a man in Little Rock (more in the section on Arkansas above). PSPP has told us that over the next two years they plan to significantly expand their efforts to promote the sustainability of policy reforms (see above). It is possible that this is an inherent risk of PSPP's model, which focuses on working directly with policymakers to achieve reforms in the near-term, but we accept this risk and believe it is outweighed by the model's apparent effectiveness in achieving policy wins.
• The risk of unintentionally increasing the crime rate. We have not seen evidence of an adverse impact on the crime rate and would guess that PSPP's work would not increase crime because of PSPP's emphasis on improving community corrections, which may reduce recidivism, and because PSPP promotes policies that focus prison beds on high level offenders.142 However, we have not investigated this issue in-depth.
• The marginal effect of additional funds to PSPP.
• PSPP's capacity for monitoring and self-evaluation is still improving.143 We would have more confidence in PSPP's track record if such measures already existed and could be examined. PSPP only tracks the accuracy of some of the intermediate-level predictions that are inputs into the forecasts it uses as evidence of impact and, as far as we know, does not always evaluate the accuracy of these forecasts.144 However, PSPP's self-evaluation strikes us as reasonable given the large number of policy changes they have been involved with in a relatively short period of time. Also, PSPP’s plans going forward include helping states establish more rigorous measurement of outcomes, including population, costs, and recidivism.
• We have not closely looked into whether some of the reduction in prison population (either in states PSPP assisted or in other states) might have merely been reallocated into county jails. However, we would be surprised if this were a problem in the states where PSPP worked because of PSPP's attention to reducing jail backlogs.145
• We have not looked into whether PSPP's reforms and the Project's broader efforts to address and reshape the crime/incarceration debate (through national research and polling, partnership with other organizations, promoting nontraditional voices for reform, and other means) have had an impact on states where PSPP did not provide intensive technical assistance. Much of this work strikes us as impactful, but we find it difficult to rigorously evaluate.

Good Ventures will be making a grant of 3,000,000 over two years to PSPP. This grant would constitute less than 25% of PSPP's biennial budget. This collaboration represents new territory for GiveWell. It is our first close collaboration with an organization whose mission is to facilitate policy change in the U.S. Because of the political nature of PSPP's mission, communications are often a substantive and sensitive element of their work. For example, framing and communications are crucial parts of the policymaking process, so we believe that writing about legislative negotiations while they are in progress could influence those negotiations. We therefore may have to be somewhat more flexible with respect to our requests for transparency than we have in the past.146 In this context, we have chosen to make our agreement with respect to information sharing, transparency, and other aspects of our collaboration more formal than our agreements with past grantees and top charities. Our overarching goal is to publicly publish our overall opinions about PSPP's progress and the impact of our grant to them, while respecting the political nature of PSPP's work and, whenever possible, transparently telling our followers about any limitations on our ability to share confidential information or our opinions. A summary of the agreement's main points follows: GiveWell and Good Ventures entered into a grant agreement with The Pew Charitable Trusts in mid-2014 to support Pew’s Public Safety Performance Project. In connection with this relationship: • GiveWell publishes commentary and updates evaluating Pew’s progress, all of which is shared with Pew in advance of posting. Substantive updates are limited to one in each of two grant periods, and one update after the end of the final grant period. • If GiveWell or Good Ventures wishes to publish comments about a federal or state policy initiative where Pew is actively working toward legislation that has not passed or regulatory changes that are not finalized, they will only publish content that has been approved in advance by Pew. • GiveWell and Good Ventures agree to respect the confidentiality of private information shared by Pew. • GiveWell and Good Ventures provide updates to Pew on all forthcoming publications, commentary, and pending grants related to criminal justice and drug policy. • Before the finalization of any pending grant whose purpose is to fund work (specifically advocacy work, a state policy initiative, or a ballot initiative) where Pew is actively working toward legislation that has not passed or regulatory changes that are not finalized, Pew will have an opportunity to provide comments to GiveWell and Good Ventures. • The parties communicate regularly via phone, email, and, when possible, in person to discuss progress. • The Agreement can be terminated prior to its expiration, unilaterally by any of the parties to it, in which case any uncommitted grant funds will be returned, and GiveWell and Good Ventures will refrain from further public commentary on the project. This agreement was in place at the time of this page's publication and - in accordance with our usual procedures - PSPP was able to review and privately comment on an outline of the page and on the page itself before it was published. ## Sources Document Source Adam Gelb, PSPP Director, comments on GiveWell outline Unpublished Adam Gelb, PSPP director, email to GiveWell on January 13, 2014 Unpublished Adam Gelb, PSPP Director, email to GiveWell on January 17, 2014 Unpublished Adam Gelb, PSPP Director, email to GiveWell on January 8, 2014 Unpublished GiveWell's non-verbatim summary of a conversation with Adam Gelb, PSPP Director, on August 23, 2013 Source GiveWell's non-verbatim summary of a conversation with Gara LaMarche on October 31, 2013 Source GiveWell's non-verbatim summary of a conversation with Grant Tennille on January 17, 2014 Source GiveWell's non-verbatim summary of a conversation with Jay Neal on January 15, 2014 Source GiveWell's non-verbatim summary of a conversation with Seema Gajwani on August 28, 2013 Source GiveWell's non-verbatim summary of a conversation with Steve Teles on June 12, 2013 Source GiveWell's non-verbatim summary of conversations with Mark Schmitt on May 22 and June 14, 2013 Source "Inmate spike delays report", Arkansas News Source (Archive) Jake Horowitz and Brian Elderbroom conversation on January 10, 2014 Unpublished Jennifer Laudano, PSPP communications director, email to GiveWell on November 14, 2013 Unpublished LaVigne et al 2014 Source (Archive) "Lawmakers ask prison officials for suggestions on how to improve system", Arkansas News Source (Archive) Meeting with Adam Gelb, Nicole Surber, and Michael Caudell-Feagan on January 28, 2014 Unpublished Pew. One in 100: Americans Behind Bars in 2008 Source (Archive) PSPP comments on draft of this report Unpublished PSPP conversation on December 19, 2013 Unpublished PSPP presentation to Good Ventures and GiveWell Unpublished PSPP website Unpublished PSPP. 2011 Kentucky Reforms Cut Recidivism, Costs Source (Archive) PSPP. 2012 Georgia Public Safety Reform Source (Archive) PSPP. AR Impact Assumptions Unpublished PSPP. AR impact calculations Unpublished PSPP. Arkansas's 2011 Public Safety Reform Source (Archive) PSPP. Description of JRI Unpublished PSPP. Leading on Public Safety: 4 Governors Share Lessons Learned from Sentencing and Corrections Reform Source (Archive) PSPP. Memo regarding comparing prison population trends across states Source PSPP. Prison populations and forecasts spreadsheet Unpublished PSPP. South Carolina's Public Safety Reform Source (Archive) Sentencing and corrections reforms in justice reinvestment states Source (Archive) Urban Institute South Carolina Case Study Source (Archive) • 1. http://www.pewstates.org/projects/public-safety-performance-project-328068 • 2. PSPP comments on draft of this report • 3. In addition, Delaware received assistance from the Vera Institute of Justice through the Justice Reinvestment Initiative (JRI), PSPP's partnership with the U.S. Department's Bureau of Justice Initiative, but not from PSPP itself. Adam Gelb, PSPP Director, comments on GiveWell outline • 4. • 5. • 6. PSPP website • 7. Adam Gelb, PSPP Director, comments on GiveWell outline • 8. "The Pew Public Safety Performance Project works at both the national and state levels. Nationally, the work involves a significant amount of original research. . ." GiveWell's non-verbatim summary of a conversation with Adam Gelb, PSPP Director, on August 23, 2013 • 9. • 10. "Nationally, the work involves a significant amount of research on topics including: • 11. "To expand on the broad goals described above, over the next two years the project will seek to accomplish the following goals in both its adult and juvenile work: . . . engage and educate influential constituencies and nontraditional allies." PSPP presentation to Good Ventures and GiveWell • 12. • 13. "Public opinion has voiced a preference for reform, including conservative voices organized through Right on Crime . . . . Pew's national public opinion polls have also revealed these preferences. The public support gets the attention of policy makers and demonstrates that the public approves of a broader set of actions than policy makers might have expected." GiveWell's non-verbatim summary of a conversation with Adam Gelb, PSPP Director, on August 23, 2013 • 14. • "This is something that other conservatives came to see as a conservative issue, in part because of the Texas Public Policy Foundation’s conservative credentials." GiveWell's non-verbatim summary of a conversation with Steve Teles on June 12, 2013 • "Also critically important are longtime Foundation grantees who work to advance sensible criminal justice reform among conservative audiences, such as the Texas Public Policy Foundation, Right on Crime, and Prison Fellowship Ministries [The Texas Public Policy Foundation and Prison Fellowship Ministries together form the Right on Crime Initiative]." GiveWell's non-verbatim summary of a conversation with Seema Gajwani on August 28, 2013 • "There were already major figures, including the Chief Justice, Speaker Ralston, and Governor Deal, who were fully engaged in reform but Pew helped to increase support by bringing Right on Crime, the Texas Public Policy Foundation (TPPF), the former Texas House Corrections Chairman, Jerry Madden, and others into the state. This was important for securing the support of conservative legislators, who make up about two thirds of Georgia’s legislature. Conservatives have historically been “tough on crime,” so it was helpful to have well-respected leaders such as Newt Gingrich and Jeb Bush talking about why it is important to be 'smart on crime.'" GiveWell's non-verbatim summary of a conversation with Jay Neal on January 15, 2014 • 15. • 16. PSPP. Description of JRI • 17. "This idea set out the core principles on which the current justice reinvestment model is based: an emphasis on data analysis to drive justice system decision-making and the use of cutting-edge research to promote efficiency and generate cost savings that can support the adoption and expansion of additional [evidence based practices] and reduce the strain on state budgets." LaVigne et al 2014, pg. 6. • 18. "The guiding principle of JRI is data-driven, consensus-based, bipartisan, and interbranch decisionmaking." LaVigne et al 2014, pgs. 10-11. • 19. "A state begins by convening an interbranch and bipartisan working group of policymakers and justice system leaders. This working group develops a shared vision of justice reform, guides data analysis, encourages information sharing, and engages in policy option development. It is imperative that the working group is interbranch and bipartisan, enabling it to take a system level approach to criminal justice reform that will be politically feasible. Working groups are formed through a variety of mechanisms—such as statute or appointment— or an established criminal justice committee or commission can be repurposed. In Pennsylvania, the Pennsylvania Commission on Crime and Delinquency convened the working group;25 in Louisiana, the Louisiana Sentencing Commission, established by statute, served as the state’s JRI working group.26" LaVigne et al 2014, pg. 13. • 20. "Phase 1: Data analysis and system assessment. The project analyzes the state's data in order to identify the key drivers of the state's prison population and costs. Pew also audits the major corrections programs, policies, and practices to determine whether they are aligned with evidence-based practices." GiveWell's non-verbatim summary of a conversation with Adam Gelb, PSPP Director, on August 23, 2013 • 21. "Phase 2: Policy development. Using the findings from the first stage, and based on research and experience in other states, the project customizes policy options that are responsive to the particular situation in the state, and then facilitates consensus in the state's working groups on a package of policy reforms. The project also creates simulations of the impact of various policy options on prison population and costs. For example, Pew might model how increasing the felony theft threshold would impact the justice system." GiveWell's non-verbatim summary of a conversation with Adam Gelb, PSPP Director, on August 23, 2013 • 22. "Phase 3: Legislative Support/Public Education. The project helps the state leadership move the consensus policy package through the legislature, using outreach to stakeholders and the media." GiveWell's non-verbatim summary of a conversation with Adam Gelb, PSPP Director, on August 23, 2013 • 23. • 24. "Representative Neal first became seriously engaged with criminal justice reform in 2010 when, along with several other Georgia leaders, he attended a conference on evidence-based sentencing and corrections reform that was sponsored by Pew. At the conference, attendees were introduced to examples of reforms from other states, primarily Texas. After returning from the conference, the group of leaders from Georgia continued to meet to map out a plan for reforming their own state's criminal justice system." GiveWell's non-verbatim summary of a conversation with Jay Neal on January 15, 2014 • 25. • "Pew staff analyzed data to understand what crimes most Arkansas prisoners had been charged with and how prison populations would be affected by modifying sentences for various offenses." GiveWell's non-verbatim summary of a conversation with Grant Tennille on January 17, 2014 • "To do so, the Council looked at various elements of the criminal justice system, including drivers of the prison population, types of crimes for which people were being sent to prison, and the lengths of sentences. Pew facilitated this work by providing the Council with technical assistance, access to experts, and information about evidence-based practices. For example, Pew, along with Applied Research Services, Inc. and the Department of Corrections, analyzed data showing that Georgia’s prisons and jails were at capacity, with nearly five thousand offenders waiting in county jails to be transferred to state prisons. Data projections showed that rising prison populations were expected to require264 million in expenditures over the next 5 years. This data helped convince the Council of the importance of making changes to Georgia’s corrections system." GiveWell's non-verbatim summary of a conversation with Jay Neal on January 15, 2014
• 26.
• 27.
• 28.
"Over the months prior to the 2011 legislative session, Pew helped the working group set up multiple meetings to bring together stakeholders and familiarize them with criminal justice reform. Some stakeholders who supported reform during the working group stage began to question their support once they saw the bill in writing. Mr. Tennille worked to reach agreement among stakeholders, especially prosecutors, so that the bill would be passed." GiveWell's non-verbatim summary of a conversation with Grant Tennille on January 17, 2014
• 29.
"Pew conducted a lot of stakeholder outreach and education in support of reform. There were already major figures, including the Chief Justice, Speaker Ralston, and Governor Deal, who were fully engaged in reform but Pew helped to increase support by bringing in organizations such as Right on Crime, the Texas Public Policy Foundation (TPPF), the former Texas House Corrections Chairman, Jerry Madden. This was important for securing the support of conservative legislators, who make up about two thirds of Georgia’s legislature. Conservatives have historically been “tough on crime,” so it was helpful to have well-respected leaders such as Newt Gingrich and Jeb Bush talking about why it is important to be “smart on crime.” Pew also engaged the Georgia Public Policy Foundation [a state-based conservative think tank that is the sister organization of TPPF] and reached out to local media to share research and submit op-eds, which resulted in a lot of positive media coverage." GiveWell's non-verbatim summary of a conversation with Jay Neal on January 15, 2014
• 30.
"Pew also engaged the Georgia Public Policy Foundation [a state-based conservative think tank that is the sister organization of TPPF] and reached out to local media to share research and submit op-eds, which resulted in a lot of positive media coverage." GiveWell's non-verbatim summary of a conversation with Jay Neal on January 15, 2014
• 31.
"As negotiations proceeded, Pew answered technical questions and advised the bill's supporters about the impact of various alternative proposals on the projected prison population. " GiveWell's non-verbatim summary of a conversation with Grant Tennille on January 17, 2014
• 32.
"Two Pew staff members with policy expertise spent significant time in Arkansas, making themselves available to legislators who had questions about the bill." GiveWell's non-verbatim summary of a conversation with Grant Tennille on January 17, 2014
• 33.

Sentencing and corrections reforms in justice reinvestment states

• 34.

• 35.

The "limited" reforms occurred in Arizona (2008), Illinois (2009), Louisiana (2011), Alabama (2011), and Missouri (2012). PSPP presentation to Good Ventures and GiveWell. PSPP does not include Delaware in either of these groups.

• 36.

PSPP presentation to Good Ventures and GiveWell

• 37.

• 38.
• 39.

Meeting with Adam Gelb, Nicole Surber, and Michael Caudell-Feagan on January 28, 2014

• 40.
• 41.
We chose Arkansas, fairly arbitrarily from a list of five states (Arkansas, Georgia, Hawaii, South Carolina, and South Dakota) for which PSPP had provided us with impact forecasts. Jennifer Laudano, PSPP communications director, email to GiveWell on November 14, 2013. We considered the fact that Arkansas was a medium-sized Southern state in which PSPP claimed a substantial victory.
• 42.
PSPP forecast that Arkansas' reform bill would save 3,202 prison beds in 2020. 457 of these beds (14%) were saved by implementing earned discharge from parole, 777 (24%) were saved by encouraging adherence to sentencing guidelines and 1,136 (35%) were saved by implementing parole release guidelines. PSPP. AR impact calculations. Thus a total of (457+777+1,136)/3,202 = 74% of the beds saved were due to these three policies.
• 43.
• "When offenders are sent to prison they are given a transfer eligibility date (TED) indicating when, absent a disciplinary record or failure to satisfy special conditions, they will be transferred to parole supervision.17 The working group’s analysis indicated that more than half of inmates are held, on average, six months beyond their presumptive TED. No reason is recorded for most of these delays, which accounted for nearly 1,200 prison beds in 2009.18" PSPP. Arkansas's 2011 Public Safety Reform, pg. 5.
• "Improve Parole Release Process: Develop and implement parole release guidelines with a presumption of release for offenders without a disciplinary record. Begin transfer proceedings earlier (6 months prior to release) and streamline case review of offenders housed in county jails with roving intake teams. Implement mandatory training requirements and appointment qualifications for the Parole board and require it to produce monthly performance reports." PSPP. Arkansas's 2011 Public Safety Reform, pg. 9.
• 44.
"Focus on High-Risk Offenders by Permitting Earned Discharge from Probation and Parole. Grant DCC authority to discharge offenders at ½ of their community supervision term if they have complied with court-ordered conditions. Require DCC to convene a group of judges, prosecutors, public defenders and law enforcement officials to determine criteria for discharge and provide notification to the prosecuting attorney or parole board before discharge." PSPP. Arkansas's 2011 Public Safety Reform, pgs. 8.
• 45.
"Strengthen Reporting Requirements and Encourage Compliance with the Sentencing Guidelines. Merge the sentencing guideline worksheet and departure form with the sentencing and commitment form and require signature from the sentencing judge and prosecuting attorney. Enhance research capacity at the Sentencing Commission and require it to produce annual reports on guideline compliance (including county-level data)." PSPP. Arkansas's 2011 Public Safety Reform, pgs. 8-9.
• 46.
"Early vs. later states: Evaluating the impact of JRI reforms on prison populations necessarily requires emphasis on states that adopted reforms further in the past. It is too soon to gauge results in the most recent states such as Kansas, South Dakota, Oregon, and West Virginia since those reforms have been in effect for less than a year. It is important to note, however, that the JRI process has evolved substantially since it was first introduced in 2007. The more recent states have received a more robust package of technical assistance and, in part because of that, have enacted more comprehensive reforms which, if well implemented, would produce larger population impacts than were achieved in the earlier states." PSPP. Memo regarding comparing prison population trends across states, pgs. 3-4.
• 47.
• 48.
Some policies supported by PSPP that we would guess would reduce recidivism include:
• "Risk and needs assessments, implemented in 16 JRI states, help predict a person’s risk to reoffend through the identification of criminal risk factors. These assessments inform decisions about detention, incarceration, and release conditions as well as the allocation of supervision and treatment resources. . . .
• Intermediate and graduated sanctions establish swift and certain responses, such as short jail stays, for parole and probation technical violators. These sanctions are alternatives to reincarceration. The HOPE (Hawaii Opportunity Probation with Enforcement) model for probationers, which couples swift and certain punishment with drug testing, is being piloted in three JRI states. Some states have developed response matrices that include both punitive and incentive-based responses designed to promote offender accountability and positive behavior change. Fifteen JRI states adopted intermediate and graduated sanctions.
• Community-based treatment programs were developed or expanded in 11 JRI states. States expanded the availability of programming and services by increasing funding for key services such as substance abuse treatment, and many encourage the use of these programs by requiring that reentry plans be developed for exiting prisoners. . . .
• Mandatory supervision requirements ensure that certain exiting prisoners receive post-release supervision. States may use risk assessments to target serious offenders or those at high risk of reoffending for supervision. This type of policy change was adopted by seven states." LaVigne et al 2014, pg. 2.
• 49.
• 50.
"Total state spending on corrections— including bonds and federal contributions—topped $49 billion last year, up from$12 billion in 1987." Pew. One in 100: Americans Behind Bars in 2008, p. 11. We have not vetted this data.
• 51.
"While there is wide variance among states, in 2008 prisons cost our 33 surveyed states an average of about $79 per inmate per day—or almost$29,000 per year." 1 in 31: The Long Reach of American Corrections, p. 12.
• 52.

"In addition, states are assessed for their overall need for JRI, with a particular focus on those states that have had large increases in corrections populations and costs. On the basis of these selection criteria, the TA provider prepares a recommendation for BJA and Pew outlining the state’s strengths as a JRI site and potential barriers to JRI implementation." LaVigne et al 2014, pg. 11.

• 53.

In earlier years, these forecasts were created by PSPP's contractors. More recently, PSPP's staff has created these forecasts internally. PSPP conversation on December 19, 2013. For simplicity, we refer to the forecasts as PSPP's throughout.

• 54.
• Additionally, for Arizona's 2008 bill, PSPP shared its baseline forecast of what would have occurred in the absence of reform but no post-reform projection was made in that state.
• The Vera Institute of Justice provided the technical assistance to Alabama, Delaware, Illinois, and Louisiana's reforms and did not create projections for those reforms. Adam Gelb, PSPP Director, comments on GiveWell outline We did not request projections from South Carolina (because we had already observed graphs of those projections, which are published in LaVigne et al 2014, pg. 115). We do not know why PSPP did not provide projections from Arizona, Connecticut, Michigan, Oklahoma, Pennsylvania's 2008 reform, or Wisconsin.
• Calculated from PSPP. Prison populations and forecasts spreadsheet.
• 55.
• Some minor judgment calls were made when determining the "five-year" forecast for each state because for most states we only had annual data, most reforms took place mid-year, and we did not have data on the effective date of each reform (many reforms may have been implemented over time and may not have had a single "effective date.")
• 56.
Calculated from PSPP. Prison populations and forecasts spreadsheet.
• 57.
"The new law is projected to save Arkansas $875 million in averted prison construction and operation costs through 2020." PSPP. Arkansas's 2011 Public Safety Reform, pg. 1. • 58. "HB 1176 is projected to avert all the anticipated growth in prison population and costs over the next five years, saving at least$264 million. " PSPP. 2012 Georgia Public Safety Reform, pg. 1.
• 59.
"The changes will allow Hawaii to bring back inmates housed in mainland prisons and save the state an estimated $130 million over five years." PSPP. Leading on Public Safety: 4 Governors Share Lessons Learned from Sentencing and Corrections Reform, pg. 6. • 60. "The state estimates the reforms will save$422 million over 10 years, allowing increased investment in programs to reduce recidivism with residual funds available for state budget relief." PSPP. 2011 Kentucky Reforms Cut Recidivism, Costs, pg. 1.
• 61.
"S. 1154 reduces the need to build and operate new prison beds during the next five years by 1,786, saving the state a projected total of \$241 million during the next five years." PSPP. South Carolina's Public Safety Reform
• 62.
We chose Arkansas from a list of five states for which PSPP sent us impact projections. Arkansas was chosen fairly arbitrarily, although we did consider that it was a mid-sized, Southern state for which PSPP claimed fairly high impact.
• 63.
• 64.
• For each policy change that PSPP modeled, it estimated the number of newly admitted inmates to whom the policy change applies in the first year of the model and the average reduction in time served for each such inmate. The reduction in time served is multiplied by the number of inmates to whom the policy applies to get the number of beds saved by the policy. By subtracting the number of beds saved from the baseline forecast of prison population, PSPP calculates a forecast of the post-reform prison population. For years after the model's first year, PSPP models the number of inmates admitted to whom each policy applies as growing by 1.6%. This depends on an (unrealistic) assumption that the relative effects of each policy change stay the same over time (i.e. that the number of admitted inmates to whom each policy applies grows at the same rate). PSPP also makes adjustments to avoid double counting prisoners to whom more than one policy applies. PSPP. AR impact calculations
• We focused on vetting Pew's estimates of the percent deviation from "the cost of doing nothing" caused by each policy change and therefore did not additionally vet the absolute value of the "cost of doing nothing" forecasts themselves.
• 65.

Data on Arkansas' prison population provided in PSPP. Prison populations and forecasts spreadsheet. (16,147-14,686)/14,686 = 9.9%.

• 66.
• 67.
• 68.

19,222-16,279 = 2,943. 2,943/19,222 = 15.3%.

• 69.
• 70.
Jake Horowitz and Brian Elderbroom conversation on January 10, 2014.
• 71.
PSPP does not believe this will be a large effect because many prisoners do not serve more than a year in prison (hence the bed is saved in the same year in which they are admitted). Jake Horowitz and Brian Elderbroom conversation on January 10, 2014. In general, we did not feel PSPP explained why the model was designed in this way but we did not push hard for an answer and believe it was reasonable that they did not know this detail three years after the model was constructed by one of PSPP's contractors.
• 72.
By contrast, PSPP's newer microsimulations used to forecast impacts in more recent states explicitly accounts for the possibility that offenders whose length of stay in prison is reduced by a policy change have some likelihood of returning to prison. PSPP conversation on December 19, 2013
• 73.

PSPP forecasted that, in the bill's absence, Arkansas' prison population would have been 16,767 in 2011. Instead, due to reform, they forecast a prison population of 14,226. PSPP. Prison populations and forecasts spreadsheet. 16,767-14,226 = 2,541 saved prison beds.

• 74.
• "Parole Release Guidelines/Parole Release From Jail: Assumes percentage of offenders held beyond TED date could be reduced to 25% and held on average 3 months before release. Comparing with the current 55% held over for an average of 6.2 months generates impact." PSPP. AR Impact Assumptions
• " When offenders are sent to prison they are given a transfer eligibility date (TED) indicating when, absent a disciplinary record or failure to satisfy special conditions, they will be transferred to parole supervision.17 The working group’s analysis indicated that more than half of inmates are held, on average, six months beyond their presumptive TED. No reason is recorded for most of these delays, which accounted for nearly 1,200 prison beds in 2009." PSPP. Arkansas's 2011 Public Safety Reform, pg 5.
• We rely here on data provided by PSPP and did not carefully vet their process for collecting this data.
• 75.
• 76.
Jake Horowitz and Brian Elderbroom conversation on January 10, 2014
• 77.
• "Parole Release Guidelines/Parole Release From Jail: Assumes percentage of offenders held beyond TED date could be reduced to 25% and held on average 3 months before release. Comparing with the current 55% held over for an average of 6.2 months generates impact." PSPP. AR Impact Assumptions
• PSPP told us that estimating the impact of these policies was as much of an art as a science and told us that they relied strongly on advice from criminal justice practitioners on the ground in forming their estimates.
• (.55*6.2 - .25*3)/(.55*6.2) = 78%
• 78.
• PSPP estimated that parole release guidelines/parole release from jail would lead to 967 beds saved in 2011. PSPP. AR impact calculations
• 967/16,767 baseline projected prisoners in 2011 = 5.8%
• 79.
• "One section of Arkansas’s 2011 reform package sought to streamline the process of transfer to parole supervision by establishing presumptive parole absent disciplinary infractions, requiring parole board training, and ensuring transparency in parole board decisions. Results: Prior to reform, 55 percent of inmates were held beyond their transfer eligibility date for an average of 6.2 months each, accounting for nearly 1,200 prison beds. Six months after reform, the percent of inmates held beyond their eligibility date had fallen to 47 percent for an average of 4.2 months each." Adam Gelb, PSPP Director, email to GiveWell on January 17, 2014.
• Predicted reduction in prison beds devoted to prisoners held beyond their transfer eligibility dates: (.55*6.2 - .25*3)/(.55*6.2) = 78%
• Reduction in prison beds devoted to prisoners held beyond their transfer eligibility dates as of six months after reform: (.55*6.2 - .47*4.2)/(.55*6.2) = 42%
• Progress toward PSPP's prediction: 42/78 = 53.8%
• 80.
"Focus on High-Risk Offenders by Permitting Earned Discharge from Probation and Parole: Grant DCC authority to discharge offenders at ½ of their community supervision term if they have complied with court-ordered conditions. Require DCC to convene a group of judges, prosecutors, public defenders and law enforcement officials to determine criteria for discharge and provide notification to the prosecuting attorney or parole board before discharge." PSPP. Arkansas's 2011 Public Safety Reform, pg. 8.
• 81.
"The impact on the prison population was created by assuming parole violators with a new felony who violated during the last 50% of their total imposed parole sentence would no longer have additional time imposed on their new charge sentence." PSPP. AR Impact Assumptions
• 82.
"The impact on the prison population was created by assuming parole violators with a new felony who violated during the last 50% of their total imposed parole sentence would no longer have additional time imposed on their new charge sentence." PSPP. AR Impact Assumptions
• 83.
• "In 2009 there were 2,236 admissions to prison for parole violations with a new charge and 29% of them violated in the latter 50% portion. These offenders would have their total LOS reduced by 10.3 months." PSPP. AR Impact Assumptions Note that this data was provided to us by PSPP and we did not carefully vet the way the data was collected.
• 2,236 admissions for a parole violation with a new charge * 20% [sic, should be 29%] violating in the second half of their term and therefore no longer on parole under the new policy = 447.2 parolees who would not have been revoked if the policy had been implemented in 2009. 447.2*1.016^2 = 462 such parolees once those admissions are scaled up to account for growth between 2009 and 2011. 462*10.3 month average length of revocation/12 months per year = 396 prison beds saved. PSPP. AR impact calculations
• PSPP's model appears to have a slight typo leading to a small underestimate of the number of beds saved due to this policy. Their description of their assumptions says that 29% of admissions for parole violations with a new charge occurred in the second half of the parolees' parole term, but the model itself multiplies by .20 instead of .29. We do not know whether the typo was in PSPP's description of the assumptions behind its model or in the model itself.
• 84.
396 beds saved by earned discharge from parole/16,767 baseline forecasted prison population in 2011 = 2.4%
• 85.
While parole officers technically have discretion over whether to allocate good time to well-behaved parolees, leaving open the possibility that this policy could be undermined by practitioners on the ground, PSPP has told us that parole board policy now instructs parole officer to allocate good time unless a parolee misbehaves and that parole officers are unlikely to deviate from this policy. Jake Horowitz and Brian Elderbroom conversation on January 10, 2014
• 86.
• 87.
Jake Horowitz and Brian Elderbroom conversation on January 10, 2014
• 88.
• "A second impact assumed a 20% reduction in total net sentences for offenders in levels 1-5. No diversions we assumed in this impact. 20% was determined to be the level total sentences in level 1-5 were at in 2002." PSPP. AR Impact Assumptions
• As a benchmark, PSPP also estimated the effect of complete adherence to sentencing guidelines. "Complete adherence to sentencing guidelines consists of 2 impacts. Complete diversion from prison if an offender was sent to prison but fell within a cell where prison was not an option. The second impact was created by comparing the total net sentence an offender received with the recommended range for the appropriate cell. In the event an offender was sentenced over the presumptive range, the offender was assigned the maximum end of the presumptive range. In the event an offender was sentenced under the presumptive range, the offender was assigned the minimum of the presumptive range. The combination of these 2 calculations results in the total impact." PSPP. AR Impact Assumptions. Complete adherence to guidelines would save 4,004 prison beds in 2011. PSPP. AR impact calculations.
• 89.
674 beds saved/4,004 beds saved by complete adherence = 16.8%.
• 90.

"The Public Safety Improvement Act passed both chambers of the General Assembly with overwhelming bipartisan majorities and was signed into law by Governor Beebe in March 2011." PSPP. Arkansas's 2011 Public Safety Reform, pg. 1.

• 91.
• 92.

“Wendy Ware of Denver-based JFA Associates [PSPP's contractor] told lawmakers the firm needed another six months — until March — to assess an increase of about 1,400 prison inmates since June." "Inmate spike delays report", Arkansas News.

• 93.
• "Prison Director Ray Hobbs told legislators the prison population has risen in the past month. He said one factor is a rising number of parolees being locked up until their court dates because of recent policy changes by the state Board of Corrections. . . .The prison board approved the . . . policies and regulations in July to address concerns over the Department of Community Correction’s handling of a parolee Darrell Dennis, who has been of accused of committing murder while free." "Lawmakers ask prison officials for suggestions on how to improve system", Arkansas News
• "Dennis was last released from the Pulaski County jail on May 8 after parole officials decided not to revoke his parole pending a mental evaluation. Less than 48 hours later, 18-year-old Forrest Abrams of Fayetteville was found dead at a Little Rock intersection. On May 22, Dennis was arrested and charged in the slaying. His parole was revoked June 5." "Inmate spike delays report", Arkansas News
• 94.

Urban Institute South Carolina Case Study

• 95.

In particularly, based on eyeballing graphs of projections and actual data, the projections seemed:

• Substantially optimistic once (Missouri)
• Moderately optimistic three times (Kansas 2007, Kentucky, Ohio)
• Mildly optimistic twice (Vermont, Arkansas based on our knowledge of events in 2013)
• Very accurate four times (Hawaii, New Hampshire, Pennsylvania, and Texas)
• Moderately pessimistic four times (Rhode Island, North Carolina, Georgia, and South Carolina)

The projection of post-reform prison populations in Nevada was also a substantial overestimate, but because the actual data in no way resembles the projection that prison populations would continue increasing after reform, we would guess this was because of external factors and not because Nevada's reforms were substantially more effective than predicted.

• 96.
• Note that the first set of graphs is very similar to those provided above by PSPP. However, it differs in X respects
• These graphs only include actual data (not forecasts).
• These graphs include states where PSPP worked but for which PSPP did not provide data.
• These graphs cover a slightly different time span than those provided by PSPP (2002 to 2012) for each state.
• These graphs use a different data source – the U.S. Bureau of Justice Statistics' Corrections Statistics Analysis Tool – while PSPP collected data from the states themselves. See the next bullet for more detail on this data.
• The data from these graphs and throughout this analysis comes from the Bureau of Justice Statistics' Corrections Statistics Analysis Tool and is the total number of prisoners within each state's jurisdiction. The data for certain states includes inmates held in nonsecure privately operated community corrections facilities and juveniles held in contract facilities, jail populations, prisoners in the custody of the state instead of prisoners under that state's jurisdiction, or dual jurisdiction cases. We would guess that these minor differences in the ways that states report data to the Bureau of Justice Statistics would not account for major differences in the prison population trends in these states. We have uploaded a spreadsheet with detailed notes on the data here.
• We used Sentencing and corrections reforms in justice reinvestment states to determine which states underwent reform and in which year.
• The graph of "unreformed" states includes Florida and New York. See below for a discussion of potentially idiosyncratic changes that contributed to decreases in incarceration in those states. If Florida and New York were excluded, the decline in prison population would be less pronounced, but the pattern of population growth subsiding in 2007-2008 would remain.
• The prison population indices for each state were created by calculating the percent change between the prison population in that year and the prison population on December 31 of the year before reform and then adding 100. This creates an index set to 100 on Dec. 31 of the year before reform.
• 97.

Despite a compelling case when this graph is viewed in isolation, we do not include Arkansas because, as discussed above, prison populations returned to their prior levels in 2013.

• 98.

We do not emphasize the average change between the year before reform and five years after reform because only three states have such data.

• 99.
• ". . . . In addition, states are assessed for their overall need for JRI, with a particular focus on those states that have had large increases in corrections populations and costs. On the basis of these selection criteria, the TA provider prepares a recommendation for BJA and Pew outlining the state’s strengths as a JRI site and potential barriers to JRI implementation." LaVigne et al 2014, pg. 11
• "JRI selects states for project participation based, in part, on the potential for individual states to achieve meaningful reforms that will protect public safety, hold offenders accountable, and control corrections costs. One of the major motivating factors for state leaders to consider reform has been the significant cost of projected prison growth in their states. So high projected growth is one of the factors considered in selecting JRI states. This introduces substantial “bias” into comparisons of JRI and non-JRI states. Even if there were similarities among groups of states on many of the crucial system and political dimensions, this selection bias means that JRI states in general were facing steeper growth curves than other states. That JRI states tend to have fast-growing prison populations is illustrated by the GiveWell analysis: some of the states with the highest post-2008 growth indices (South Dakota, West Virginia, and Idaho) have since become JRI states." PSPP. Memo regarding comparing prison population trends across states, pg. 1.
• 100.
See charts of prison population indices for states that underwent comprehensive reform and states that did not participate in justice reinvestment below.
• 101.
• 102.
• "New York’s population dropped because of a huge drop in felony arrests in NYC; California’s prison count is down because of the federal court order." Adam Gelb, PSPP director, email to GiveWell on January 13, 2014
• Meeting with Adam Gelb, Nicole Surber, and Michael Caudell-Feagan on January 28, 2014
• "State justice systems and criminal penalty laws vary tremendously. This variation has enormous impact on the two determinants of prison populations: admissions and length of stay. The political environment in states is vastly different as well. In some states, top elected officials are unanimous in their desire for and commitment to reform; in others, there is no consensus for change. Beyond leadership, there are varying levels of interest in reform among other influential policy makers and stakeholders. Since these myriad and in many cases unquantifiable state-specific factors help drive prison populations, a simple comparison of prison trends in one group of states with another group can raise questions but offer no answers for why the trends are similar or different. For instance, the fact that the New York prison population dropped due to the reduction in felony arrests in New York City has no bearing on what happened to the prison populations in Georgia or North Carolina or why. If California radically reduces its prison population under order from the federal courts, that does not diminish (or enhance) the importance of comprehensive reform efforts in Oregon or Ohio, which did not face such a mandate. The differences among states are dramatized by the fact that in 2012, four years after the national prison population peaked, several states were posting significant population drops while prison counts still continued to climb in nearly half (24) of the states. PSPP. Memo regarding comparing prison population trends across states, pg. 1
• 103.
"There may be some national trends in policy making by governors and legislators, and in case-level decision making by individual judges, prosecutors, parole board members, probation officers and other system actors, that are influenced by an overall climate of reform and general attitudes toward crime and punishment. While Pew, JRI partners and others working in the field cannot claim anything near exclusive credit for creating an environment for reform, changing the national conversation from “tough vs. soft on crime” to “public safety return on investment” is a key goal of our broader work. This work has included hundreds of research publications, extensive media outreach, and scores of convenings with leading policy makers across the country. These national efforts “contaminate” the group of non-JRI states by exposing policy makers in those states to justice reinvestment data and strategies. Comparing them to the intensive assistance states effectively counts these broader national efforts against the impact of work in intensive JRI states. This is unfortunate since any policy change effort should seek to maximize change with available resources." PSPP. Memo regarding comparing prison population trends across states, pg. 2.
• 104.
PSPP. Memo regarding comparing prison population trends across states, pg. 2.
• 105.
"Beyond these broad national impacts, Pew and its partners have provided limited assistance in some states, including California, Florida and Colorado. Counting these states among the non-JRI group further complicates comparison of JRI vs. non-JRI states. California, for instance, picked up the Performance Incentive Funding (PIF) concept from 2008 JRI work in Arizona and fashioned it with very light assistance from PSPP into SB 678 of 2009. In its first year of implementation, the California PIF program cut the number of probation revocations to state prison by nearly 6,200, a nearly the identical number by which the state’s prison population dropped from 2009 to 2010." PSPP. Memo regarding comparing prison population trends across states, pg. 2.
• 106.
• The following states are included in this graph: Delaware, Georgia, Hawaii, Missouri, Oklahoma, Alabama, Arkansas, Kentucky, Louisiana, North Carolina, Ohio, Michigan, New Hampshire, South Carolina, Illinois, Wisconsin, Arizona, Connecticut, Pennsylvania, Rhode Island, Vermont, Kansas, Nevada, Texas.
• This graph combines data from states that participated in JRI in various years. Therefore growth in states that received assistance in more recent years may offset the reductions in growth in states that received assistance in earlier years. This affects the shape of the curve and interpretation of the impact of the reforms.
• 107.
• The following states are included in this graph: Oregon, South Dakota, West Virginia, Alaska, California, Colorado, Florida, Idaho, Indiana, Iowa, Maine, Maryland, Massachusetts, Minnesota, Mississippi, Montana, Nebraska, New Jersey, New Mexico, New York, North Dakota, Tennessee, Utah, Virginia, Washington, Wyoming.
• Three of the states -- Oregon, South Dakota, and West Virginia – participated in JRI in 2013. We would not expect the effects to be visible in this graph.
• 108.
• The graph of "states Pew worked in by 2012" plots the sum of the prison population in all such states. The graph of "states Pew did not work in" plots the sum of the prison population in all other states.
• PSPP suggested to us that the disruption in the trend of prison growth in states in which they did not operate may largely be explained by idiosyncratic policy changes in two large states – New York and California – and therefore may not suggest that prison growth would have stopped in other states without PSPP's intervention. Additional graphs exploring this hypothesis can be found here. While New York and California do account for a large portion of the decline in prison populations in states where PSPP did not work, prison growth in those states stagnated even when New York and California are excluded.
• 109.

For the two graphs of indices, we first created an index of the prison population in each state, which was set to 100 in 2008. Details of this calculation can be seen in our spreadsheet posted here.

The "comprehensive reform index" graph plots the mean index in each year among states that PSPP categorized as achieving "comprehensive reform" by 2012. The "non-Pew index" graph plots the mean index in each year among states that PSPP did not work in by 2012. The two "index" graphs are a measure of changes in prison populations that are not weighted by the size of each state.

• 110.
The following states are included in this graph: Georgia, Hawaii, Oklahoma, Arkansas, Kentucky, North Carolina, Ohio, Michigan, New Hampshire, South Carolina, Wisconsin, Connecticut, Pennsylvania, Rhode Island, Vermont, Kansas, Nevada, Texas.
• 111.
The following states are included in this graph: Oregon, South Dakota, West Virginia, Alaska, California, Colorado, Florida, Idaho, Indiana, Iowa, Maine, Maryland, Massachusetts, Minnesota, Mississippi, Montana, Nebraska, New Jersey, New Mexico, New York, North Dakota, Tennessee, Utah, Virginia, Washington, Wyoming, Delaware, Missouri, Alabama, Louisiana, Illinois, Arizona.
• 112.

Meeting with Adam Gelb, Nicole Surber, and Michael Caudell-Feagan on January 28, 2014

• 113.
Colorado passed these reforms with the facilitation and assistance of the Colorado Criminal Justice Reform Coalition and the Independence Institute, two organization supported by PSPP. Adam Gelb, PSPP Director, comments on GiveWell outline
• 114.
• "Mr. Tennille was one of two Deputy Chiefs of Staff to Arkansas Governor Mike Beebe during the efforts that resulted in the passage of SB 750, Arkansas' criminal justice reform bill. The majority of Mr. Tennille’s involvement came after the bill was introduced. Mr. Tennille helped write the State of the State speech in which Governor Beebe introduced criminal justice reform as a legislative priority. After this speech, a prosecuting attorney and acquaintance of Mr. Tennille’s called him to say that there was unrest among prosecutors about the Governor’s mention of criminal justice reform. Mr. Tennille had built relationships with many prosecuting attorneys in Arkansas at a prior job as a reporter for a statewide newspaper, so he became the point person within the Governor’s office to reach out to prosecutors and gain their support." GiveWell's non-verbatim summary of a conversation with Grant Tennille on January 17, 2014
• " Jay Neal is a former representative in the Georgia State House of Representatives. While serving in the Georgia House, he was vice-chair of the Public Safety Committee and Chairman of the House Committee on State Institutions and Property. . . . Representative Neal co-sponsored HB 1176, a bill that significantly reformed Georgia's criminal justice system. " GiveWell's non-verbatim summary of a conversation with Jay Neal on January 15, 2014
• 115.

"PSPP retained Cranford Johnson Robinson Woods (CJRW) to support the state’s criminal justice reform effort through government and public relations work. Mr. Tennille’s wife served as CJRW’s lead staff person." GiveWell's non-verbatim summary of a conversation with Grant Tennille on January 17, 2014

• 116.

"In that same year, Governor Deal was elected; he was a strong proponent of criminal justice reform from the start of his term." GiveWell's non-verbatim summary of a conversation with Jay Neal on January 15, 2014

• 117.

"Representative Neal first became seriously engaged with criminal justice reform in 2010 when, along with several other Georgia leaders, he attended a conference on evidence-based sentencing and corrections reform that was sponsored by Pew." GiveWell's non-verbatim summary of a conversation with Jay Neal on January 15, 2014

• 118.
• 119.

"To do so, the Council looked at various elements of the criminal justice system, including drivers of the prison population, types of crimes for which people were being sent to prison, and the lengths of sentences. Pew facilitated this work by providing the Council with technical assistance, access to experts, and information about evidence-based practices. For example, Pew, along with Applied Research Services, Inc. and the Department of Corrections, analyzed data showing that Georgia’s prisons and jails were at capacity, with nearly five thousand offenders waiting in county jails to be transferred to state prisons." GiveWell's non-verbatim summary of a conversation with Jay Neal on January 15, 2014

• 120.

GiveWell's non-verbatim summary of a conversation with Jay Neal on January 15, 2014

• 121.

"This was a reversal of the attitude that had been prevalent in Georgia for many years before. The shift was due to the strength of the case for reform being led by Gov. Deal and demonstrated by the data that Pew had helped the committee present in the report and discuss with legislators. " GiveWell's non-verbatim summary of a conversation with Jay Neal on January 15, 2014

• 122.

"In collaboration with the Council and the Governor’s legislative staff, Pew played a critical role in designing and drafting the reforms of HB 1176, HB 242, and HB 349. A couple of Pew’s employees spent significant time in Georgia working closely with the Council co-chairs. Pew staff also did clerical and legal work and was in frequent email communication with Council members as the bills were being drafted. One of Pew's strengths was that, although they did a significant amount of work, they ensured that state policymakers maintained control of the Council's decisions. Pew also brought in the Crime & Justice Institute to provide assistance throughout the process and with the legislation. . . . Pew conducted a lot of stakeholder outreach and education in support of reform. There were already major figures, including the Chief Justice, Speaker Ralston, and Governor Deal, who were fully engaged in reform but Pew helped to increase support by bringing in organizations such as Right on Crime, the Texas Public Policy Foundation (TPPF), the former Texas House Corrections Chairman, Jerry Madden. This was important for securing the support of conservative legislators, who make up about two thirds of Georgia’s legislature. Conservatives have historically been “tough on crime,” so it was helpful to have well-respected leaders such as Newt Gingrich and Jeb Bush talking about why it is important to be “smart on crime.”" GiveWell's non-verbatim summary of a conversation with Jay Neal on January 15, 2014

• 123.
"There was some criticism from the media that proponents of reform were being “soft on crime,” and some district attorneys and sheriffs believed that the state was trying to shift a greater burden to the counties. Pew helped the Council conduct breakout groups to facilitate the participation of sheriffs and district attorneys in the reform process and to identify their concerns. Pew also helped the Council address these concerns and gain law enforcement support. The Sheriff’s Association was not initially in favor of reform, but became supportive after a number of components were added to the bill. For example, HB 1176 enacted electronic transmission of sentencing packages, which increased the speed with which offenders were transferred from county to state responsibility, and set the maximum stay at probation detention centers at 180 days. The Georgia Dept. of Corrections, in response to the recommendations of the Council expanded the Residential Substance Abuse Treatment program, adding 600 additional beds for those who had violated their parole or probation." GiveWell's non-verbatim summary of a conversation with Jay Neal on January 15, 2014
• 124.

"Before Pew began working on criminal justice reform in Georgia, the issue was not on the agenda. It’s not clear what would have happened if Pew had never gotten involved; it’s probable that the Governor, the Speaker, and the Chief Justice would have championed reforms, but there was no outspoken advocate to the General Assembly prior to Pew’s engagement and the Council’s formation." GiveWell's non-verbatim summary of a conversation with Jay Neal on January 15, 2014

• 125.

"It’s not clear what would have happened if Pew had never gotten involved; it’s probable that the Governor, the Speaker, and the Chief Justice would have championed reforms, but there was no outspoken advocate to the General Assembly prior to Pew’s engagement and the Council’s formation. Representative Neal led criminal justice reform in the state legislature, and said that he would not have taken up that role if not for the initial conference and continuing engagement with Pew." GiveWell's non-verbatim summary of a conversation with Jay Neal on January 15, 2014

• 126.

"Representative Neal recalled that the first time he heard anyone in the legislature mention the need for criminal justice reform was in a conversation with the Speaker of the House just after the Pew-sponsored conference." GiveWell's non-verbatim summary of a conversation with Jay Neal on January 15, 2014

• 127.
• "Pew is not assisting the Council with the reentry work, which initially caused some serious challenges. The Council knew that there were barriers in housing, employment and education for former inmates, but coming up with appropriate ways of addressing these problems was challenging. Initially, members of the Council tended towards the areas in which they had prior interest and experience and had difficulty prioritizing among these areas. The Council ultimately decided to contract with Dennis Schrantz, a consultant with experience in re-entry policy around the country. Working with a consultant who had an outside perspective helped Council members step outside of their particular areas of interest and compare Georgia’s existing policies to evidence-based practices." GiveWell's non-verbatim summary of a conversation with Jay Neal on January 15, 2014
• We learned from a conversation with PSPP that this consultant was referred to Georgia by PSPP. Meeting with Adam Gelb, Nicole Surber, and Michael Caudell-Feagan on January 28, 2014
• 128.

"Even if the Governor had tried to make criminal justice reform a legislative priority, it would have been very difficult without Pew’s advocacy and support. The cost of putting together a team to do the research and analysis required to produce an in-depth study similar to what Pew produced would have been prohibitive for the government, especially in a tight economic time. Pew’s experience with other states and familiarity with the data were vital to the effort. Eventually, some sort of criminal justice reform would likely have passed in Pew's absence, but it may have been much weaker than what was actually accomplished." GiveWell's non-verbatim summary of a conversation with Jay Neal on January 15, 2014

• 129.

"The majority of Mr. Tennille’s involvement came after the bill was introduced. Mr. Tennille helped write the State of the State speech in which Governor Beebe introduced criminal justice reform as a legislative priority. After this speech, a prosecuting attorney and acquaintance of Mr. Tennille’s called him to say that there was unrest among prosecutors about the Governor’s mention of criminal justice reform. Mr. Tennille had built relationships with many prosecuting attorneys in Arkansas at a prior job as a reporter for a statewide newspaper, so he became the point person within the Governor’s office to reach out to prosecutors and gain their support." GiveWell's non-verbatim summary of a conversation with Grant Tennille on January 17, 2014

• 130.

"Governor Beebe had similar views, but saw reform as necessary from a financial perspective." GiveWell's non-verbatim summary of a conversation with Grant Tennille on January 17, 2014

• 131.

"Pew's main role was providing technical assistance. Pew staff analyzed data to understand what crimes most Arkansas prisoners had been charged with and how prison populations would be affected by modifying sentences for various offenses. This enabled Pew to help the working group prioritize reforms." GiveWell's non-verbatim summary of a conversation with Grant Tennille on January 17, 2014

• 132.

"Once reforms were prioritized, Pew staff wrote an outline of a bill that would implement the desired reforms." GiveWell's non-verbatim summary of a conversation with Grant Tennille on January 17, 2014

• 133.

"Over the months prior to the 2011 legislative session, Pew helped the working group set up multiple meetings to bring together stakeholders and familiarize them with criminal justice reform. Some stakeholders who supported reform during the working group stage began to question their support once they saw the bill in writing. Mr. Tennille worked to reach agreement among stakeholders, especially prosecutors, so that the bill would be passed. As negotiations proceeded, Pew answered technical questions and advised the bill's supporters about the impact of various alternative proposals on the projected prison population. Two Pew staff members with policy expertise spent significant time in Arkansas, making themselves available to legislators who had questions about the bill. . . . Outside of these roles, Pew did not play a significant role in advocacy for the bill." GiveWell's non-verbatim summary of a conversation with Grant Tennille on January 17, 2014

• 134.

"Prior to Pew’s involvement, the main advocates for criminal justice reform in Arkansas were the Black Caucus and their allies in the Democratic Party. Though these members were pro reform, many hesitated to support the proposed legislation because they felt it should do more to address certain issues, such as sentencing disparities and reentry. Reform advocates negotiated with the Black Caucus, arguing that no reform at all was likely to pass if they were to hold out for their ideal bill and that it was better to pass a more modest bill than nothing at all. The bill's supporters also emphasized that the proposed reforms would cause better data to be collected on imprisonment than had previously existed, and this would make future conversations about reform possible. There was not a concrete proposal on the table prior to Pew's engagement and the Black Caucus did not have enough members to be able to move a bill forward on its own. Black Caucus members were not uniformly ambivalent about the bill; one of their most prominent members was Representative Darrin Williams, who co-sponsored SB 750 and helped gain the support of other Caucus members. " GiveWell's non-verbatim summary of a conversation with Grant Tennille on January 17, 2014

• 135.

"Pew staff provided the technical assistance needed to analyze massive data sets and reconcile them with hundreds of lines of criminal code. Without Pew, the government would not have had the human resources to complete this work within the same time frame. If Pew had not been involved at all, a criminal justice reform bill would probably have been proposed, but it would not have been data driven in any way; someone would have just gone through the existing legislation and made cuts fairly haphazardly, without regard for how it would actually affect prison populations." GiveWell's non-verbatim summary of a conversation with Grant Tennille on January 17, 2014

• 136.

We plan to have at least one more conversation, seeking out the point of view of someone who believes that incarceration rates should be lower but has opposed the justice reinvestment process.

• 137.
• 138.

780,000*.11 = 85,800

• 139.

(85,800 fewer prisoners at any point in time)*(10 year impact of reform)*(100% credit to PSPP) = 858,000.

• 140.
"[T]he partners are ramping up the more rigorous policy evaluations necessary to gain a deeper understanding of whether and why specific reforms succeeded or failed. These evaluations will include quantitative analyses of data and qualitative components that assess implementation and help provide valuable context." PSPP. Memo regarding comparing prison population trends across states, pg. 1.
• 141.