Program outcomes themselves (in terms of placement and retention rates and wages) are not sufficient to evaluate our applicants; as we explain in our
overview, we believe that individual motivation is important, and that some of those who are placed by our applicants could have found work without help (or with much less help).
This isn't to say that focusing on the most motivated clients is inappropriate. Screening clients for motivation can help an organization spend its limited resources where they will help as many people as possible. But when
comparing a more selective program to a less selective one, it would be misleading simply to look at which one places more people in jobs.
Here we use census data as a
very rough “comparison group” for two of our applicants, and derive estimates of how many of the people these charities serve would likely be able to attain similar incomes on their own.
How we obtained census data
We used the
InfoShare website to access 2000 census data for New York City, with the following steps:
- Log in or Subscribe (individual subscriptions are free).
- Select “Create your own two-way table.”
- Select “Public Use Micro-sample (2000 census)” for New York City
- Select the "Population" category
- Apply filters to narrow down the population based on age, ethnicity, and educational status (see below)
- Pull "Wage or Salary Income in 1999" for the filtered sample
Limitations of census data
Census data is very far from providing truly appropriate comparison groups; there are many reasons for this, including:
- To “match” census data to client data, we used only a few simple characteristics: age, ethnicity, and education level (as well as restricting our analysis to NYC). We did not control for job skills, background, etc. Clients of the charities we study could differ systematically in many ways, the most obvious being the mere fact of their coming to a nonprofit organization for help (this implies both that they have a difficult financial situation and the ability/motivation to participate in a job training program).
- The census data we used looks at the New York metropolitan area as a whole, while our applicants likely draw clients disproportionately from areas near them.
- Census data is from the year 2000, whereas most data from our applicants is much more recent. Changes in the labor market are addressed only through our use of a rough and basic wage inflation adjustment.
We are not seeking to use this data for precise comparisons and measurements of impact; rather, we use it as a reference point, to give us some understanding of what young people with little education can and do earn. More on our specific use is below.
Choice of organizations for this analysis
We believe this data is a somewhat appropriate reference point for programs such as
VFI and
Year Up, which serve young people whose main obstacle to employment seems to be their low level of education (often lacking a high school degree in VFI's case; having only a high school degree or GED in Year Up's case).
We would be much more hesitant to use it as a reference point for those serving older populations. The mere fact that someone is unemployed or underemployed, and seeking help from a charity, at the age of 30 indicates a lot about them that is not captured in Census data; by contrast, we would intuitively expect most young and undereducated people to be without strong job prospects and to look for help.
Year Up
Year Up's clients (details
here) are:
- 18-24 years old
- Predominantly African-American or Latin-American
- High school diploma or GED required
- Have not attended school since February 1, 2000 or before (the census was conducted in April 2000)
Our “comparison group” is constructed from
InfoShare using the following criteria:
- 18-24 years old
- African-American or Latin-American
- Highest level of education: high school diploma or GED
- Have not attended school since February 1, 2000 or before (the census was conducted in April 2000)
Since Year Up aims to place clients in jobs paying about $20/hr, we looked at the proportion of our comparison group that made $37,500/yr in 1999, which is slightly more than $40,000 in 2007 dollars (see
below). 4% of census participants meeting the above criteria made this much or more.
VFI
VFI's clients (details here) are:
- 18-21 years old
- Predominantly African-American or Latin-American
- No GED or high school degree required
Our “comparison group” is constructed from InfoShare using the following criteria:
- 18-21 years old
- African-American or Latin-American
- No high school diploma or GED
- Have not attended school since February 1, 2000 or before (the census was conducted in April 2000)
Since VFI aims to place clients in jobs paying about $10/hr, we looked at the proportion of our comparison group that made $17,500/yr in 1999, which is slightly more than $20,000 in 2007 dollars (see below). 18% of census participants meeting the above criteria made this much or more.
Details of wage inflation adjustment
We accessed historical wage data for the jobs VFI and Year Up prepare their graduates for using the
Occupation Employment Statistics report for the New York metropolitan area from the
Bureau of Labor Statistics. You can access specific years or occupations
here or download all tables
here.
We pulled data for the jobs whose descriptions seem most similar to those that VFI and Year Up prepare their clients for. In Year Up's case, this was Computer Support Specialist, which experienced cumulative wage inflation, according to this data, of 14% from 1999-2007. These are the numbers we roughly used to adjust wage data, although the census only allows us to look at income ranges (we can't be precise about the cutoff) so our adjustments were relatively rough.
