These wrongly arrested black men say a California bill would let police misuse face recognition

This story was originally published on The Markup. Read the original here, or lea esta historia en Español.

This article is co-published with CalMatters, a nonprofit, nonpartisan media venture explaining California policies and politics. 

In 2019 and 2020, three Black men were accused of, and jailed for, crimes they didn’t commit after police used face recognition to falsely identify them. Their wrongful arrest lawsuits are still pending, but their cases bring to light how AI-enabled tools can lead to civil rights violations and lasting consequences for the families of the accused.

Now all three men are speaking out against pending California legislation that would make it illegal for police to use face recognition technology as the sole reason for a search or arrest. Instead it would require corroborating indicators.

The problem, critics say, is that a possible face recognition “match” is not evidence — and that it can lead investigations astray even if police seek corroborating evidence.

After a contentious hearing today, the Senate Public Safety Committee unanimously voted to advance Assembly Bill 1814. It had cleared the Assembly last month without opposition.

Such a bill “would not have stopped the police from falsely arresting me in front of my wife and daughters,” Robert Williams told CalMatters. In 2020, Detroit police accused Williams of stealing watches worth thousands of dollars — the first known instance of false arrest involving face recognition in the United States — after face recognition matched a surveillance video to a photo of Williams in a state database. Investigators put his photo in a “six-pack lineup” with five others, and he was chosen by a security guard who had seen a surveillance image but not the theft itself.

“In my case, as in others, the police did exactly what AB 1814 would require them to do, but it didn’t help,” said Williams, who is Black. “Once the facial recognition software told them I was the suspect, it poisoned the investigation. This technology is racially biased and unreliable and should be prohibited.

“I implore California lawmakers to not settle for half measures that won’t actually protect people like me.”

But the bill’s author, Democratic Assemblymember Phil Ting of San Francisco, maintained that because it bans face recognition technology from being the sole criteria for a warrant, search or arrest, it would prevent wrongful apprehensions such as those in Detroit.

And he stressed that it would improve the status quo for Californians.

“Law enforcement agencies in the state do not need any permission from anyone to do anything on facial recognition right now,” Ting said. “Nothing in any state law provides guidance in that particular area. 

“This actually takes a good first step to really provide some security, to provide some civil rights protections, and to ensure that we take the first step to regulate facial recognition technology.”

The first face recognition searches in the United States took place more than two decades ago. It’s a process that begins with a photo of a suspect typically taken from security camera footage. Face recognition on your iPhone is trained to match your photo, but the kind used by law enforcement agencies searches databases of mug shots or drivers license photos can contain millions of photos, and can fail in numerous ways. Tests by researchers have shown that the technology is less accurate when attempting to identify people with dark skin, people who are Asian, Native American, people who identify as transgender, if a probe image of a suspect is low quality, or if the image in a database is outdated.

After a computer assembles a list of possible matches from a database of images, police pick a suspect from an array of candidates, then show that photo to an eyewitness. Although people tend to think they’re good at it, eyewitness testimony is a leading cause of wrongful convictions in the United States.

Because prosecutors use face recognition to identify possible suspects but ultimately rely on eyewitness testimony, the technology can play a role in a criminal investigation but remain hidden from the accused and defense attorneys.

Directives not to treat a possible match by a face recognition system as the sole basis for an arrest sometimes don’t make a difference — they failed to do so, for instance, in the case of  Alonzo Sawyer, a man who was falsely arrested near Baltimore and spent nine days in jail.

Njeer Parks, who spent nearly a year fighting allegations that he stole items from a hotel gift shop in New Jersey and then nearly hit a police officer with a stolen vehicle, came out in opposition to the California bill in a video posted on Instagram last week. The police “are not going to do their job if the AI is saying ‘It’s him’ already. That’s what happened to me.”

“I got lucky,” he told CalMatters in a phone interview about a receipt that exonerated him and kept him out of prison. “I don’t want to see anybody sitting in jail for something they didn’t do.”

Testifying at today’s hearing was the attorney for Michael Oliver, a third Black man who was wrongly accused of assaulting a high school teacher in Detroit in 2020. “The warrant request in Michael’s case was based entirely on a supposed (face recognition technology) match and a photo lineup,” said attorney David Robinson. “Other than a photo lineup, the detective did no other investigation. So it’s easy to say that it’s the officer’s fault, that he did a poor job or no investigation. But he relied on (face recognition), believing it must be right. That’s the automation bias this has been referenced in these sessions.

“So despite a warning to the officer — ‘investigative lead only’ — that prescription was trumped by the mesmerizing effect of this machine that the officer saw as faster and smarter than he, and it must be right.” 

Supporters of Ting’s bill include the California Faculty Association and the League of California Cities. The California Police Chief Association argues that face recognition can reduce criminal activity and provide police with actionable leads, and that such technology will be important as California looks to host international events such as the 2026 World Cup and the 2028 Summer Olympics in Los Angeles.

“Across the country, real-world examples of law enforcement using (facial recognition technology) to solve major crimes showcases just how important this new technology can be towards protecting our communities,” the California Police Chiefs Association has argued. It cited cases in which it says face recognition played a role in identifying the guilty, including a newspaper headquarters shooting in Maryland and a rape in New York.

Face recognition alone should never lead to false arrests, Jake Parker with the Security Industry Association told members of the California Assembly a few weeks ago. That’s why AB 1814 is meant to corroborate investigative leads with evidence, not just a possible face recognition match.

“There’s a clear need to bolster public trust that this technology is being leveraged accurately, lawfully, and in an effective way that’s also limited and non-discriminatory in a way that benefits our communities,” he said. “So we believe AB 1814 will help bolster this trust and for that reason we urge you to support this bill in its current form.”

“By allowing police to scan and identify people without limitation, AB 1814 will also increase unnecessary police interactions that too often have the potential to escalate into fatal encounters. This will remain true regardless of how accurate face recognition technology becomes,” the organizations said in a letter. “There is no way for people to find out if facial recognition is used against them and no mechanism to make sure the police comply with the law.”

Ting also authored a 2019 bill that initially placed a permanent ban on police use of body camera footage with face recognition. That was amended to a temporary ban, which ended in January 2023.

He told CalMatters he’s uncomfortable with the fact that California currently has no limits on how law enforcement agencies use face recognition. 

He said in a statement that his bill “simply requires officers to have additional evidence before they can proceed with a search, arrest, or affidavit for a warrant. I believe having a precautionary step can help protect people’s privacy and due process rights, while still allowing local governments to go further and pursue their own facial recognition bans.”

Ting’s city of San Francisco became the first major city in the nation to ban face recognition in 2019, but an analysis by City Attorney David Chiu found that the city’s passage of Proposition E in March allows police to perform face recognition searches on imagery captured by cameras and drones. The Washington Post last month reported that San Francisco police go around restrictions by requesting that law enforcement in neighboring cities conduct the search for them.

Recalled San Francisco district attorney Chesa Boudin says there have almost certainly been false arrests associated with use of face recognition in California but they would remain unknown to the public unless prosecutors filed charges and the accused later went to trial with a civil lawsuit seeking damages. Often such cases would be settled out of court.

“We absolutely need a legislative, regulatory framework for these technologies, but I don’t think AB 1814 is adequate in terms of protecting civil liberties or providing meaningful guardrails or safeguards for the use of these new and powerful technologies,” said Boudin, who now directs UC Berkeley’s Criminal Law & Justice Center.

Lawmakers have until the end of the legislative session in August to decide whether to pass  AB 1814.