Technology
| Analysis
9 June 2020
By Chris Stokel-Walker
M4OS Photos / Alamy
Software giant IBM has announced that it is withdrawing certain face-recognition products from the market, and has called for a “national dialogue” about the technology’s use by US law enforcement agencies.
The move, which comes as global protests against racism and police brutality enter their third week, is a marked change in the company’s approach to face recognition. In November 2019, IBM said “blanket bans on technology are not the answer to concerns around specific use cases”.
Flaws in face-recognition technology are well documented. A 2018 study by researchers at the Massachusetts Institute of Technology and Microsoft found that dark-skinned women are misidentified by such systems 35 per cent of the time.
Advertisement
“Face-recognition systems have internal biases because they are primarily trained on libraries of white faces,” says Alison Powell, a data ethicist at the Ada Lovelace Institute, UK. “They don’t recognise black and brown faces well.”
But the solution isn’t to train the system on a more diverse data set, says Powell. “It increases surveillance on black and brown people.”
IBM’s decision, laid out in a letter from CEO Arvind Krishna to members of the US Congress, has been reported by some outlets as an end to all of its face-recognition business, but this may not be the case.
Yoav Goldberg at the Allen Institute for AI Israel points out that the letter says IBM no longer offers “general purpose” software to analyse faces, potentially leaving it wiggle room to sell more tailored systems. IBM declined to comment for this article, saying Krishna’s letter stands on its own.
Whatever the specifics, the high-profile announcement could have a broader impact on the use of face recognition, says Frederike Kaltheuner, a tech policy fellow at the Mozilla Foundation.
Earlier this year, a draft policy document suggested the European Union was considering a five-year ban on the technology, though the final published version removed this proposal. “If a company like IBM feels that face recognition is too problematic for them, this should put the idea of a moratorium back on the political agenda in Europe,” says Kaltheuner.
A number of US cities, including San Francisco, Seattle and Oakland, have already banned the use of face recognition by police, and a police reform bill introduced by US Democrats this week would ban using the technology on footage taken by body-worn cameras.
The backlash will put pressure on companies like Amazon, which supplies police forces in the US with face-recognition services, to rethink their approach. In 2018, the American Civil Liberties Union showed that Amazon software incorrectly identified 28 faces in police mugshots as being members of Congress, with the false matches being disproportionately people of colour.
“Face recognition has taken a hit in the last year from a civil liberties perspective, breaching privacy, and the high risks of racial and gender-based discrimination,” says Nikita Aggarwal at the Oxford Internet Institute, UK. That may mean the systems are no longer commercially viable, she says.
More on these topics:
