Memo: Reducing recidivism through fair algorithms
Date: 11/20/2021
Date: 11/20/2021
Dear Mayor,
In 2015, the average cost to keep an individual incarcerated in the state of Pennsylvania was $42,727 1. Combined with the social cost of separating families and creating gaps in communities, local and federal government have a shared responsibility of supporting programs which reduce incarceration and recidivism rates through allocating resources appropriately. Our team has developed a decision making tool, which, coupled with the job training re-entry program and your support, will greatly increase the efficiency of the current systems in place.
To preface, the job training re-entry program is an effective and imperative step for individuals being released from prison. This program matches ex-offenders with job placements, offers training referrals, career counseling, provides job listings and similar employment-related services that are otherwise widely inaccessible for individuals with limited alternative support. Our program costs on average $10,000 per person, and is proven to decrease the risk of recidivism from 60% to 25% on average.
Placing ex-offenders in steady employment that matches their interests and abilities is an important effort. Preparing recently released ex-offenders with job training and practical work skills not only reduces rates of recidivism, but additionally improves the social, physical and mental health of communities who are disproportionately victimized by the American judicial system, specifically low income and African American communities.
When we allow populations within our society to be disproportionately imprisoned, we uphold discriminatory systems that prevent vulnerable communities from meeting basic standards of living. Recidivism is a risk that comes with the release of any previously incarcerated person, and as we know, that risk is higher within some populations over others. Figure 1 illustrates the observed rates of recidivism by race observed in Broward County, Florida.
We often hear that the exclusion of race from a predictive algorithm will result in an unbiased one. That is not the case, as exemplified by the COMPAS recidivism prediction algorithm built by Northpointe. In 2016, journalists from ProPublica evaluated the results of this widely used algorithm, and discovered clear division in errors made by the model across different race groups2. While the algorithm was technically accurate over all races, the model did not generalize well across race groups, creating a cascading disparate impact.
Our team has built a model inspired by improving the COMPAS algorithm, which accounts for the bias that is baked into the data. We have used COMPAS data from Broward County to train and test our algorithm. Figure 2 displays the results of our algorithm with a 50% decision threshold, and compares how the model performs across African American, Caucasian and ‘Other’ racial categories. The ‘Rate_FP’ metric stands for the rate of false positives predicted by the model. In other words, the ex-offender was predicted to recidivate but in reality did not. As you can imagine, if protocol is to keep prisoners who are expected to re-offend in jail, false positives the African American community will continue to see higher rates of incarceration and recidivism if not met with adequate support. At a 50% threshold, our model predicts more African Americans to recidivate when they in fact do not, over any other race, which is a symptom of biased data. We urge you to consider using this tool to allocate support towards incarcerated individuals who are at higher risk of re-offending, therefore lowering the risk.
To make this program more cost efficient, we are able to reduce the decision threshold from 50 to 40%, which as you can observe in figure 3, impacts the balance of our output metrics. This threshold adjustment will close the gap of errors between racial groups, while only slightly affecting the algorithm’s overall accuracy. The costs to the city associated with each prediction metric are as follows.
True Negative: Individual is predicted to not recidivate and does not in reality. These individuals should have no problem re-entering society and thus are not priority for receiving the job-training program. Individuals who are not expected to recidivate may apply for the program ~ estimated 10%.
True Positive: Individual is predicted to recidivate and does in reality. The cost associated with this outcome is the price per individual of the job-training program, which is $10,000 per person. With a 75% success rate, 25% of these predictions will recidivate which is a $42,727 cost per person.
False Negative: Individual is predicted to not recidivate but does in reality. The cost associated with this outcome is the price per individual to be in prison which is $42,727 per person.
False Positive: Individual is predicted to recidivate but does not in reality. The cost associated with this outcome is the price per individual of the job-training program, which is $10,000 per person. This cost would be better allocated towards the true positive outcomes.
With these costs in mind, we can better interpret figures 2 and 3 in terms of their associated cost/benefit. Although the overall accuracy of the model decreases, we are able to increase the overall amount of false positives which have the lowest associated cost per person. Decreasing the threshold also increases the rate of True Positives. While the associated cost of this outcome his higher, the rate of ex-offenders having exposure to the job-training program increases, therefor increasing the associated social benefit. The cost/benefit table below calculates the estimated costs associated with each prediction metric at each threshold. By lowering the threshold, we are able to reduce funds spent on the program by roughly $5,000,000.
| Variable | Count | Cost.50 | Cost.40 | Social.Benefit | Description |
|---|---|---|---|---|---|
| True_Negative | 472 | 639000 | 472000 | Individual re-enters society and becomes a productive member of local economy without government program assistance | Individual is predicted to not recidivate and does not in reality. |
| True_Positive | 556 | 7636335 | 10109053 | Individual receives job training and career support and has a 75% chance of not recidivating. Community gains a productive member of local economy | Individual is predicted to recidivate and does in reality. |
| False_Negative | 147 | 12091741 | 6280869 | NA | Individual is predicted to not recidivate but does in reality. |
| False_Positive | 365 | 1980000 | 3650000 | Individual receives job training and career support and has a 100% chance of not recidivating. Community gains a productive member of local economy | Individual is predicted to recidivate but does not in reality. |
Total Cost 50% Threshold: $22,347,076
Total Cost 40% Threshold: $17,349,648
Difference: $4,997,428
Mayor, we urge you to take into consideration utilizing our fair predictive algorithm for allocating resources towards the job training re-entry program. The future of this algorithm includes further analysis to develop varying thresholds to control for each racial group, resulting in even more equitable outcomes. This is the potential and the future of these powerful predictive tools, and you have the power to implement these tools to incite positive change in our city.
Henrichson, Christian (2017). Vera.org Vera Institute of Justice 2017.
Angwin, Julia, Jeff Larson, Surya Mattu, Lauren Kirchner. Machine Bias: There’s software used across the country to predict future criminals. And it’s biased against blacks ProPublica, May 23, 2016
Aos, Steve, Phipps, Polly, Barnoski, Robert and Lieb, Roxanne. (2001). The Comparative Costs and Benefits of Programs to Reduce Crime. Olympia: Washington State I