Risk Assessments Used in Criminal Justice Systems Too Often Clouded By Bias, Report Says

As states and cities consider how to end reliance on cash bail, officials are turning to risk assessment tools to help them decide who should be released before trial. But a new report says that the tools that have been deployed across the country are flawed because they incorporate biased data and produce unclear results.  

A majority of states use risk assessment tools during some stage of criminal justice proceedings, said Alice Xiang, the primary researcher on the study. Judges may use the tools when making pretrial detention decisions and in sentencing hearings, and many corrections departments also use them for decisions about parole.

“It’s very easy to think of these tools as neutral decision makers because they reflect numeric values,” Xiang said. “But that doesn’t mean that the data is free of bias, or that the human interpretation of it is accurate.”

The report focuses on the many courts in the U.S. that use algorithms specifically in the pretrial context to assess the risk of a defendant skipping their trial. In these systems, factors about a defendant, including their age, criminal history, employment status, and ties to the community are often weighed against historical data for those similar to them in order to develop a “risk score” that assists judges in determining whether someone should be released or detained prior to their court date.

But the report says that the use of group-level data pollutes a process that should be about the individual defendant and the alleged crime, and raises questions about the tool’s constitutionality. “[Because] societal biases are reflected in criminal justice data...it is likely that decisions made by risk assessment tools are driven in part by what protected class an individual may belong to, raising significant Equal Protection Clause concerns,” the report says.

Much of the historical data from people in the criminal justice system considered similar to the defendant is rough, Xiang said. If those people missed a court date, their reasons for skipping vary widely by circumstance, for example. “If someone misses their hearing due to issues with public transit or because they’re caring for a dependent, that is a very different reason than someone who is trying to evade justice,” she said. “If we want to make fair predictions, intentions have to factor into the assessment.”

Xiang also said judges lack the training to properly interpret the probability given to them. When a defendant is given a “high risk” categorization with a 60% chance of reoffending, it often isn’t clear whether that number represents a probability between 55% and 65%, with a mean of 60%, or if the tool computed a probability between 30% and 90%, with a mean of 60%.

The report was prompted by the passage of California’s Senate Bill 10, which eliminated cash bail in the state and replaced it with a risk assessments—but provided no guidance to localities as to how they should create those systems. Xiang said the concern there is that the tools require an investment of resources at the state and local levels to collect data and test it for biases, train judges and other stakeholders in the use of risk assessment tools, and create transparency measures so that researchers and the general public can understand how the tools are used.

The growing debate around risk assessments surfaced this week in Philadelphia, where the city wants to use the tool as part of a major criminal justice reform effort. The development of the algorithm now faces scrutiny, however, after District Attorney Larry Krasner and Chief Public Defender Keir Bradford-Grey officially withdrew their support, citing concerns about a lack of transparency and the likelihood that the system will reinforce racial biases. But frustrated city leaders, including the mayor, are arguing that no major jurisdiction has eliminated the use of cash bail, a primary goal of the reform initiative, without implementing a risk assessment system.

A jurisdiction with one of the longest histories of using risk assessments is Washington, D.C., which implemented their system to help make pretrial release decisions in 1967. Leslie Cooper, director of the D.C. Pretrial Services Agency, says that while the city is aware of the national conversation around the potential for bias in these systems, her office has found them to be useful—if they are developed and administered properly.  

“Our tool is scientifically validated and was specifically developed for D.C., which is key to ensuring that the system is fair and equitable,” Cooper noted. The risk assessment looks at 70 factors to predict a defendant’s risk of failure to appear at any of the required court appearances before their trial date, as well as their risk of being rearrested for any reason, including for dangerous or violent offenses.

Cooper says the risk assessment tool’s success can be partially attributed to the D.C. mandate that release should be the default, and detention should be the exception. The tool therefore never recommends preventative detention, but rather notes when someone is eligible for detention, leaving the ultimate decision to the court.

Xiang supports the use of risk assessment tools for auto-release, as opposed to auto-detainment, and recognizes that many places want to use risk assessment as a way to improve equity. “Most of the states’ motivations for incorporating these tools are positive. They recognize the bias in our current system and they want to push for evidence-based policy,” Xiang said. “But when we looked into our technical and ethical questions, we realized that no state is building models that adequately address all the concerns we raised.”

For those with risk assessment tools in place, Xiang suggests that jurisdictions go through a critical examination and revision process, a huge part of which must be public conversations. “A lot of this sounds highly technical, but it’s actually about policy decisions,” she said. “The difference between a high and a low risk score is about where a community finds a balance between wrongfully convicting and inaccurately letting people go. That’s not a technical question—that’s a community consensus.”

When asked about what other jurisdictions should do when considering risk assessment tools, Cooper’s suggestions largely mirror Xiang’s. “You should consider your population, and have your tool independently normed and validated against your specific demographics. D.C. does not trend with other communities with respect to criminal justice, but we only found that out through very specific research,” Cooper said.

Cooper also recommended strong transparency efforts so that the community has a chance to ask questions about the system and explore its implications. Finally, Cooper suggested educating judges, prosecutors, and defense attorneys about risk assessment tools so that they understand how the tools interact with a broader system that involves human discretion at every step.

“We share the concern of many that bias exists in these systems and we are hypersensitive to issues around race and equity, as most of the people arrested in the District are African American,” Cooper said. “But we absolutely see this tool as a positive. Over 90% of our defendants are released either on their first or second appearance in court, and 90% of those released make every court appearance after that. This proves that you can administer a fair system of pretrial release that accounts for public safety and is beneficial for communities.”